Identifying consumers in a transaction via facial recognition

ABSTRACT

A merchant and a user register with a payment processing system, which establishes a facial template based on a user image. The user signs into a payment application via a user computing device, which receives an identifier from a merchant beacon device to transmit to the payment processing system. The payment processing system transmits facial templates to the merchant camera device for other users who are also signed in to the payment application in range of the merchant beacon device. The merchant camera device compares a captured facial image against the received facial templates to identify the user. A merchant POS device operator selects an account of the user. The merchant POS device transmits transaction details to the payment processing system, which processes the transaction with an issuer system. The payment processing system receives an approval of the transaction authorization request and transmits a receipt to the merchant POS device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/701,517 filed Apr. 30, 2015, and entitled“Identifying Consumers In A Transaction Via Facial Recognition.” Thecomplete disclosure of the above-identified priority application ishereby fully incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to improving user convenience intransactions by identifying user accounts for use in transactions basedon facial recognition of users by a payment processing system.

BACKGROUND

When consumers make purchases at a merchant location, many methods ofconducting a transaction are available. Consumers may use many differentcards or accounts for purchases, such as gift cards, debit cards, creditcards, stored value cards, and other cards or accounts. The user accountidentifiers and other data represented by the cards may be communicatedto the merchant system via magnetic stripes, near field communicationtechnologies involving user computing devices, and other suitablemechanisms.

Current applications for conducting transactions at a merchant locationdo not provide the opportunity for the consumer to make a hands-freetransaction. Additionally, current applications require the consumer toperform actions to identify himself by providing user accountidentifiers or other data to the merchant system.

SUMMARY

Techniques herein provide computer-implemented methods to conduct ahands-free transaction with facial recognition of a user. In an exampleembodiment, a merchant registers with a payment processing system. Auser establishes an account with the payment processing system andtransmits an image of himself to the payment processing system toestablish a facial template associated with the user account. A usersigns into the payment application via the user computing device andenters the merchant system location. The user computing device receivesa merchant beacon device identifier from the merchant beacon device andtransmits the identifier to the payment processing system. The paymentprocessing system transmits facial templates to the merchant cameradevice corresponding to other users whose user computing devices are innetwork range of the merchant beacon device and who are signed in to thepayment application. The merchant camera device captures a facial imageof the user and identifies the user by comparing the captured facialimage against the received facial templates. The merchant point of saledevice operator selects an account of the user for use in a transactionfrom one or more displayed accounts of the user. The merchant point ofsale device transmits transaction details to the payment processingsystem, which generates a transaction authorization request to transmitto an issuer system associated with the user account selected for use inthe transaction. The payment processing system receives an approval ofthe transaction authorization request and transmits a receipt to themerchant point of sale device.

In certain other example aspects described herein, systems and computerprogram products to conduct a hands-free transaction with facialrecognition of a user are provided.

These and other aspects, objects, features, and advantages of theexample embodiments will become apparent to those having ordinary skillin the art upon consideration of the following detailed description ofillustrated example embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram depicting a system for conducting a hands-freetransaction with facial recognition of a user, in accordance withcertain example embodiments.

FIG. 2 is a block flow diagram depicting a method for conducting ahands-free transaction with facial recognition of a user, in accordancewith certain example embodiments.

FIG. 3 is a block flow diagram depicting a method for registering, by amerchant system, with a payment processing system and installinghardware at merchant system location, in accordance with certain exampleembodiments.

FIG. 4 is a block flow diagram depicting a method for registering, by auser, for an account with a payment processing system, in accordancewith certain example embodiments.

FIG. 5 is a block flow diagram depicting a method for establishing afacial template associated with a user account, in accordance withcertain example embodiments.

FIG. 6 is a block flow diagram depicting a method for establishing anaudio template associated with a user account, in accordance withcertain example embodiments.

FIG. 7 is a block flow diagram depicting a method for receiving, by auser computing device, a merchant beacon identifier broadcast by amerchant beacon device, in accordance with certain example embodiments.

FIG. 8 is a block flow diagram depicting a method for receiving, by acamera device, a facial template for each user in range of a merchantbeacon device, in accordance with certain example embodiments.

FIG. 9 is a block flow diagram depicting a method for receiving, by acamera device, notification from a payment processing system as usersenter or leave a network range of a merchant beacon device, inaccordance with certain example embodiments.

FIG. 10 is a block flow diagram depicting a method for initiating, by auser, a transaction at a merchant point of sale device, in accordancewith certain example embodiments.

FIG. 11 is a block flow diagram depicting a method for identifying, by acamera device, a user via facial recognition, in accordance with certainexample embodiments.

FIG. 12 is a block flow diagram depicting a method for identifying, by apayment processing system, a user via voice recognition, in accordancewith certain example embodiments.

FIG. 13 is a block flow diagram depicting a method for identifying, by amerchant point of sale device operator, a user via a challenge and aresponse, in accordance with certain example embodiments.

FIG. 14 is a block flow diagram depicting a method for conducting atransaction, in accordance with certain example embodiments.

FIG. 15 is a block diagram depicting a computing machine and module, inaccordance with certain example embodiments.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

The example embodiments described herein provide computer-implementedtechniques for conducting a hands-free transaction with facialrecognition of a user.

In an example embodiment, a merchant registers with a payment processingsystem. A merchant system installs one or more merchant beacon devices,one or more merchant point of sale devices, and one or more merchantcamera devices at a merchant system location. For example, a userestablishes an account with the payment processing system and downloadsa payment application on a user computing device associated with theuser. In an example, the user transmits an image of himself and/or anaudio recording of himself to the payment processing system to establisha facial template and/or audio template associated with the useraccount. A user enters a merchant system location and signs into thepayment application via the user computing device. The user computingdevice receives a merchant beacon device identifier from the merchantbeacon device and transmits the identifier to the payment processingsystem. The payment processing system transmits facial templates to themerchant camera device corresponding to users whose user computingdevices are in network range of the merchant beacon device and who aresigned in to the payment application. The merchant camera devicecaptures a facial image of the user and identifies the user based oncomparing the captured facial image against the received facialtemplates. Alternatively, the user submits an audio recording to thepayment processing system via the merchant point of sale device, whichidentifies the user based on comparing the received audio recordingagainst audio templates for users. In yet another embodiment, themerchant point of sale device operator identifies the user based on auser's response to a challenge. After identifying the user, the merchantpoint of sale device displays one or more accounts of the user. Themerchant point of sale device operator selects an account of the userfor use in a transaction. The merchant point of sale device transmitstransaction details to the payment processing system, which generates atransaction authorization request to transmit to an issuer systemassociated with the user account selected for use in the transaction.The payment processing system receives an approval of the transactionauthorization request and transmits a receipt to the merchant point ofsale device.

In an example embodiment, a merchant system registers with a paymentprocessing system. A merchant system operator installs a paymentapplication on a merchant point of sale device. In another example, themerchant system operator installs the payment application on a pluralityof merchant point of sale devices at a merchant system location. Amerchant beacon device receives a beacon identifier code from a paymentprocessing system. For example, the merchant system operator installsone or more merchant beacon devices at the merchant system location. Themerchant beacon device broadcasts the merchant beacon identifier codevia wireless communication at the merchant system location. The merchantsystem operator installs a merchant camera device at the merchant systemlocation to correspond to the merchant point of sale device. In anotherexample, a plurality of merchant camera devices are installed at themerchant system location, each merchant camera device corresponding to aparticular merchant point of sale device. In yet another example, aparticular merchant camera device may correspond to two or moreparticular merchant point of sale devices. The payment processing systemreceives a camera device identifier and associates it with acorresponding beacon identifier of the merchant beacon device.

In an example embodiment, the user registers with a payment processingsystem. For example, the user accesses a payment processing systemwebsite via a user computing device associated with the user. The userregisters with the payment processing system and downloads a paymentapplication onto the user computing device. In an example embodiment,the payment processing system establishes a facial template associatedwith the user account. For example, the payment application displays arequest for the user to capture a facial image via the user computingdevice. The user selects an option to capture a facial image. Thepayment application activates a camera module on the user computingdevice and the users captures a facial image of himself. The paymentprocessing system receives the facial image. The payment processingsystem creates a facial template associated with the user account basedon the received facial image. The payment processing system deletes thereceived facial image. In another example embodiment, the paymentprocessing system establishes a audio template associated with the useraccount. The payment application requests and receives user audio viathe user computing device. The payment application creates an audiotemplate associated with the user account based on the received audio ofthe voice of the user. The payment processing system deletes thereceived audio of the voice of the user.

The user signs in to a payment application on the user computing device.The user carries the user computing device within a threshold distanceof a merchant beacon device at the merchant system location. The usercomputing device receives a merchant beacon identifier broadcast by themerchant beacon device and transmits the received merchant beaconidentifier and a user account identifier to the payment processingsystem. The payment processing system receives the merchant beaconidentifier and the user account identifier. The payment processingsystem extracts a facial template associated with the user accountidentifier and identifies a merchant camera device associated with themerchant beacon device identifier.

The payment processing system transmits a facial template of theidentified user to the merchant camera device associated with themerchant beacon device identifier. For example, a facial templateassociated with the identified user's account is transmitted to themerchant camera device. The merchant camera device receives the facialtemplate of the user. The merchant camera device adds the facialtemplate of the user to a current customer log. The merchant cameradevice periodically updates the current customer log based on updatesreceived from the payment processing system. For example, the paymentprocessing system transmits a subsequent facial template of a subsequentuser that, carrying a user computing device via which the user is signedin to the payment application, enters a threshold distance of a merchantbeacon device required to establish a wireless network connection. Inthis example, the payment processing system receives the merchant beacondevice identifier transmitted by the user computing device and transmitsa facial template of the subsequent user to the merchant camera device.In another example, in response to detecting that the user computingdevice associated with a user in the current customer log is no longermaintaining a network connection with the merchant beacon device, is nolonger observing the merchant beacon device, or is no longer signed into the payment application, the payment processing system transmits anotice that a user has left a merchant location to the merchant cameradevice. In this example, the merchant camera device deletes theindicated user from the current customer log.

The user approaches a merchant point of sale device. The merchant pointof sale device operator totals items of the user for purchase. Themerchant point of sale device operator asks the user to select a paymentoption. The user directs the merchant point of sale device operator toinitiate a transaction via the payment application. For example, aspreviously discussed, the payment application is installed on both themerchant point of sale device and the user computing device. Themerchant point of sale device operator selects an option on the merchantpoint of sale device to initiate a transaction using the paymentapplication. The merchant point of sale device transmits a request toidentify the user to the merchant camera device via the paymentprocessing system. In this example, the payment processing systemcommunicates with both the merchant camera device and the merchant pointof sale device.

In another example embodiment, the merchant point of sale devicereceives an indication from the merchant camera device that the user hasbeen identified and the merchant point of sale device displays an optionfor the user to initiate a transaction using the payment application.For example, the merchant point of sale device may receive an indicationthat the user has been identified directly from the merchant cameradevice over a network. In another example, the merchant camera devicetransmits an indication that the user has been identified to the paymentprocessing system and the payment processing system then transmits theindication that the user has been identified to the merchant point ofsale device. In these example embodiments, the merchant point of saledevice operator, in response to the merchant point of sale devicedisplaying an option to initiate a transaction using the paymentapplication, asks the user whether the user would like to initiate atransaction using the payment application. In these example embodiment,the user directs the merchant point of sale device operator to initiatethe transaction via the payment application and the merchant point ofsale operator selects the corresponding option on the merchant point ofsale device user interface.

The merchant camera device captures video of the user. For example, theuser is positioned in front of the point of sale device and the merchantcamera device is positioned to be able to capture a video of the user'sface. In an example embodiment, the merchant camera device startscapturing video of the user only when the camera device receives arequest to identify the user. For example, the merchant camera devicereceives a request to identify the user from the payment processingsystem or from the merchant point of sale device. In another exampleembodiment, the merchant camera device starts capturing video when themerchant camera device receives an indication from the paymentprocessing system that a user computing device associated with the userhas established a network connection with the merchant beacon device. Inthis example embodiment, the merchant camera device does not capturevideo when there are no users with associated user computing deviceswithin network range of the merchant beacon device. The merchant cameradevice extracts a facial image of the user from the captured video andgenerates a facial template from the captured facial image. The merchantcamera device deletes the captured video and extracted facial image. Themerchant camera device retrieves facial templates from the currentcustomer log. For example, the current customer log comprises a list ofusers and associated facial templates for users associated with usercomputing devices that have currently established a network connectionwith a merchant beacon device at the merchant system location. In anexample embodiment, the current customer log comprises volatile ortransient memory. For example, the current customer log is not saved anduser information is added or deleted from the current customer log asuser computing devices associated with respective users enter or leave anetwork range of the merchant beacon device. The merchant camera devicecompares the generated facial template from the extracted facial imageto facial templates from the current customer log. The merchant cameradevice is able to identify the user if there is a match between a facialtemplate from the current customer log and the generated facialtemplate. The merchant camera device is unable to identify the user ifthere is no match between a facial template from the current customerlog and the generated facial template. If the merchant camera device isable to identify the user, the merchant camera device notifies thepayment processing system of the identity of the user. In anotherexample embodiment, if the merchant camera device is able to identifythe user, the merchant camera device notifies the merchant point of saledevice of the identity of the user.

If the merchant camera device is unable to identify the user based onfacial recognition, the merchant camera device notifies the paymentprocessing system that the user cannot be identified based on facialrecognition. In an example embodiment, if the user cannot be identifiedbased on facial recognition, the payment processing system identifiesthe user based on audio recognition. In another example embodiment, thepayment processing system does not identify users based on audiorecognition. In an example embodiment, if the payment processingidentifies users based on audio recognition, the payment processingsystem retrieves audio templates corresponding to users from the currentcustomer log. The payment processing system transmits a request to themerchant system point of sale device to record an audio of the user. Themerchant system point of sale device displays a request for the user torecord audio, records a voice input of the user, and transmits the voiceinput to the payment processing system. The payment processing systemcompares the received voice input against the retrieved audio templatescorresponding to users from the current customer log. The paymentprocessing system is able to identify the user if there is a matchbetween a audio template from the current customer log and the receivedvoice input of the user. The payment processing system is unable toidentify the user if there is no match between an audio template fromthe current customer log and the received voice input of the user.

If the payment processing system is unable to identify the user based onvoice recognition, the merchant point of sale device operator isnotified by the payment processing system to issue a challenge to theuser. The user provides a challenge response and the merchant point ofsale operator inputs the response into the merchant point of saledevice. The merchant point of sale device displays potential users basedon the challenge response. For example, the merchant point of saledevice transmits the response to the payment processing system and thepayment processing system accesses a database comprising a list or tablethat associates challenges with corresponding responses and users. Inthis example, the payment processing system identifies the user bycorrelating the challenge and the response to identify one or more usersin the database. In this example, the payment processing systemtransmits the one or more identified users to the merchant point of saledevice. In this example, the merchant point of sale device displays theone or more identified users to the merchant point of sale deviceoperator. The merchant point of sale device operator selects a user. Inan example, the merchant point of sale device operator may compare avisual image or name of the user displayed on the user computing deviceto the visual appearance of the current customer at the merchant pointof sale device and/or documentation presented by the user to themerchant point of sale operator. In an example, the merchant point ofsale device transmits the identity of the user identified by themerchant point of sale operator.

The point of sale device displays accounts of the identified user. Forexample, after receiving the identity of the user from the merchantcamera device, receiving the identity of the user from the merchantpoint of sale device, or identifying the user, the payment processingsystem transmits information associated with one or more accounts of theidentified user. The merchant point of sale device operator selects auser account for transaction and confirms the transaction withpermission of the user. The merchant point of sale device transmitstransaction details to the payment processing system. For example,transaction details may comprise a total amount of the transaction, aselected user account for use in the transaction, an account of themerchant for use in the transaction, and other useful or relevantinformation. The payment processing system transmits a transactionauthorization request to an issuer system. For example, the issuersystem is associated with the user account selected for use in thetransaction. The issuer system approves the transaction authorizationrequest and transmits a transaction authorization approval to thepayment processing system. The payment processing system transmits atransaction receipt to the merchant point of sale device.

By using and relying on the methods and systems described herein, thepayment processing system, the merchant camera device, the merchantbeacon device, the user computing device, and the merchant point of saledevice enable the user to conduct a transaction with the merchant systemwithout the user having to interact with the user computing device orproduce identity documents or physical payment cards, as required insome current technology. As such, the systems and methods describedherein may reduce the inputs required by the user via the user computingdevice and the inputs required by the merchant point of sale deviceoperator to identify the user.

Example System Architecture

Turning now to the drawings, in which like numerals indicate like (butnot necessarily identical) elements throughout the figures, exampleembodiments are described in detail.

FIG. 1 is a block diagram depicting a system 100 for conducting ahands-free transaction with facial recognition of a user 101, inaccordance with certain example embodiments. As depicted in FIG. 1, thesystem 100 includes network computing devices 110, 130, 140, 150, and160 that are configured to communicate with one another via one or morenetworks 120. In some embodiments, a user associated with a device mustinstall an application and/or make a feature selection to obtain thebenefits of the techniques described herein.

In example embodiments, the network 120 can include a local area network(“LAN”), a wide area network (“WAN”), an intranet, an Internet, storagearea network (“SAN”), personal area network (“PAN”), a metropolitan areanetwork (“MAN”), a wireless local area network (“WLAN”), a virtualprivate network (“VPN”), a cellular or other mobile communicationnetwork, Bluetooth, Bluetooth low energy, NFC, or any combinationthereof or any other appropriate architecture or system that facilitatesthe communication of signals, data, and/or messages. Throughout thediscussion of example embodiments, it should be understood that theterms “data” and “information” are used interchangeably herein to referto text, images, audio, video, or any other form of information that canexist in a computer-based environment.

Each network computing device 110, 130, 140, 150, and 160 includes adevice having a communication module capable of transmitting andreceiving data over the network 120. For example, each network computingdevice 110, 130, 140, 150, and 160 can include a server, desktopcomputer, laptop computer, tablet computer, a television with one ormore processors embedded therein and/or coupled thereto, smart phone,handheld computer, personal digital assistant (“PDA”), or any otherwired or wireless, processor-driven device. In the example embodimentdepicted in FIG. 1, the network computing devices 110, 130, 140, 150,and 160 are operated by users 101, merchant beacon device 120 operators,merchant point of sale (“POS”) device 130 operators, merchant cameradevice 140 operators, issuer system 150 operators, and paymentprocessing system 160, respectively.

An example user computing device 110 comprises an antenna 111, a Wi-Ficontroller 112, a payment application 113, a user interface 115, a datastorage unit 116, a camera module 117, a web browser 118, and acommunication application 119.

In an example embodiment, the antenna 111 is a means of communicationbetween the user computing device 110 and a merchant beacon device 120.In an example embodiment, a Wi-Fi controller 112 outputs through theantenna 111 a radio signal, or listens for radio signals from themerchant beacon device 120. In another example embodiment a Bluetoothcontroller or a near field communication (“NFC”) controller is used. Inan example embodiment, the Wi-Fi controller 112 outputs through theantenna 111 a radio signal, or listens for radio signals from thepayment card device 120.

In an example embodiment, the Wi-Fi controller 112 is capable of sendingand receiving data, performing authentication and ciphering functions,and directing how the user computing device 110 will listen fortransmissions from the merchant beacon device 120 or configuring theuser computing device 110 into various power-save modes according toWi-Fi-specified procedures. In another example embodiment, the usercomputing device 110 comprises a Bluetooth controller or an NFCcontroller capable of performing similar functions. An example Wi-Ficontroller 112 communicates with the payment application 113 and iscapable of sending and receiving data over a wireless, Wi-Ficommunication channel. In another example embodiment, a Bluetoothcontroller 112 or NFC controller 112 performs similar functions as theWi-Fi controller 112 using Bluetooth or NFC protocols. In an exampleembodiment, the Wi-Fi controller 112 activates the antenna 111 to createa wireless communication channel between the user computing device 110and the merchant beacon device 120. The user computing device 110communicates with the merchant beacon device 120 via the antenna 111. Inan example embodiment, when the user computing device 110 has beenactivated, the Wi-Fi controller 112 polls through the antenna 111 aradio signal, or listens for radio signals from the merchant beacondevice 120.

In an example embodiment, the payment application 113 is a program,function, routine, applet, or similar entity that exists on and performsits operations on the user computing device 110. In certain exampleembodiments, the user 101 must install the payment application 113and/or make a feature selection on the user computing device 110 toobtain the benefits of the techniques described herein. In an exampleembodiment, the user 101 may access payment application 113 on the usercomputing device 110 via the user interface 115. In an exampleembodiment, the payment application 113 may be associated with thepayment processing system 160. In another example embodiment, thepayment application 113 may be associated with a merchant systemassociated with the merchant beacon device 120, the merchant point ofsale device 130 and the merchant camera device 140. In yet anotherexample embodiment, two payment applications 113 exist, one associatedwith the merchant system and another associated with the paymentprocessing system 160.

In an example embodiment, the user interface 115 enables the user 101 tointeract with the payment application 113 and/or web browser 118. Forexample, the user interface 115 may be a touch screen, a voice-basedinterface, or any other interface that allows the user 101 to provideinput and receive output from an application or module on the usercomputing device 110. In an example embodiment, the user 101 interactsvia the user interface 115 with the payment application 113 and/or webbrowser 118 to configure user 101 accounts on the payment processingsystem 160. In another example embodiment, the user 101 interacts viathe user interface 115 with the payment application 113 and/or the webbrowser 118 to enable hands-free payments, if needed.

In an example embodiment, the data storage unit 116 comprises a local orremote data storage structure accessible to the user computing device110 suitable for storing information. In an example embodiment, the datastorage unit 116 stores encrypted information, such as HTML5 localstorage.

In an example embodiment, the camera module 117 may be any module orfunction of the user computing device 110 that obtains a digital image.The camera module 117 may be resident on the user computing device 110or in any manner logically connected to the user computing device 110.For example, the camera module 117 may be connected to the usercomputing device 110 via the network 120. The camera module 117 may becapable of obtaining individual images or a video scan. Any othersuitable image capturing device may be represented by the camera module117.

In an example embodiment, the user 101 can use a communicationapplication 119, such as a web browser 118 application or a stand-aloneapplication, to view, download, upload, or otherwise access documents orweb pages via a distributed network 120.

In an example embodiment, the web browser 118 can enable the user 101 tointeract with web pages using the user computing device 110. In anexample embodiment, the user 101 may access the user's 101 accountmaintained by the payment processing system 160 via the web browser 118.In another example embodiment, the user 101 may access the a merchantsystem website via the web browser 118. In certain example embodimentsdescribed herein, one or more functions performed by the paymentapplication 113 may also be performed by a web browser 118 applicationassociated with the payment processing system 160.

In an example embodiment, the communication application 119 can interactwith web servers or other computing devices connected to the network120, including the user computing device 110 and a web server of amerchant system.

In certain example embodiments, one or more functions herein describedas performed by the payment application 113 may also be performed by aweb browser 118 application, for example, a web browser 118 applicationassociated with a merchant system website or associated with the paymentprocessing system 160. In certain example embodiments, one or morefunctions herein described as performed by the payment application 113may also be performed by the user computing device 110 operating system.In certain example embodiments, one or more functions herein describedas performed via the web browser 118 may also be performed via thepayment application 113.

An example merchant beacon device 120 comprises an antenna 121 and aWi-Fi controller 122. In an example embodiment, a merchant systemlocation comprises one or more merchant beacon devices 120 installed atthe merchant system location. In an example embodiment, each installedmerchant beacon device 120 is associated by a payment processing system160 with a particular merchant camera device 140 installed at themerchant location. For example, the payment processing system 160 maycomprise a database that correlates merchant beacon device 120identifiers with merchant camera device 140 identifiers for associatedmerchant camera devices 140. For example, a merchant camera device 140identifier may comprise hardware identifier specific to the device suchas a serial number or a MAC ID. In another example, a merchant beacondevice 120 identifier may comprise a hardware identifier specific to thebeacon device or an identifier generated by the payment processingsystem 160 and stored in the merchant beacon device 120. An examplemerchant beacon device 120 is programmed to broadcast, emit, orotherwise transmit a particular merchant beacon device 120 identifierover a wireless network 120 to any user computing devices 110 within athreshold distance required to maintain the wireless network 120. Forexample, the wireless network may comprise a Wi-Fi network 120, aBluetooth network 120, an NFC network 120, or any other appropriatewireless network 120.

In an example embodiment, the antenna 121 is a means of communicationbetween the user computing device 110 and a merchant beacon device 120.In an example embodiment, a Wi-Fi controller 122 outputs through theantenna 121 a radio signal, or listens for radio signals from the usercomputing device 110. In another example embodiment a Bluetoothcontroller or a near field communication (“NFC”) controller is used. Inan example embodiment, the Wi-Fi controller 122 outputs through theantenna 121 a radio signal, or listens for radio signals from thepayment card device 120.

In an example embodiment, the Wi-Fi controller 122 is capable of sendingand receiving data, performing authentication and ciphering functions,and directing how merchant beacon device 120 will listen fortransmissions from the user computing device 110 or configuring themerchant beacon device 120 into various power-save modes according toWi-Fi-specified procedures. In another example embodiment, the merchantbeacon device 120 comprises a Bluetooth controller or an NFC controllercapable of performing similar functions. An example Wi-Fi controller 122communicates with the payment application 113 and is capable of sendingand receiving data over a wireless, Wi-Fi communication channel. Inanother example embodiment, a Bluetooth controller 122 or NFC controller122 performs similar functions as the Wi-Fi controller 122 usingBluetooth or NFC protocols. In an example embodiment, the Wi-Ficontroller 122 activates the antenna 121 to create a wirelesscommunication channel between the user computing device 110 and themerchant beacon device 120. The merchant beacon device 120 communicateswith the user computing device 110 via the antenna 121. In an exampleembodiment, when the merchant beacon device 120 has been activated, theWi-Fi controller 122 polls through the antenna 121 a radio signal, orlistens for radio signals from the user computing device 110.

An example merchant point of sale device 130 comprises an audio module131, payment application 133, a user interface 135, a data storage unit136, and a communication application 139.

In an example embodiment, the audio module 131 may be any module orfunction of the merchant POS device 130 that captures an audio input ofan external environment of the merchant POS device 130. The audio module131 may be resident on the merchant POS device 130 or in any mannerlogically connected to the merchant POS device 130. For example, theaudio module 131 may be connected to the merchant POS device 130 via thenetwork 120. The audio module 131 may be capable of obtaining an audiorecording. Any suitable audio recording device may be represented by theaudio module 131.

In an example embodiment, the payment application 133 is a program,function, routine, applet, or similar entity that exists on and performsits operations on the merchant point of sale device 130. In certainexample embodiments, the merchant point of sale (“POS”) device operator102 or other merchant system operator must install the paymentapplication 133 and/or make a feature selection on the merchant point ofsale device 130 to obtain the benefits of the techniques describedherein. In an example embodiment, the merchant POS device operator 102may access the payment application 133 on the merchant POS device 130via the user interface 135. In an example embodiment, the paymentapplication 133 may be associated with the payment processing system160. In another example embodiment, the payment application 133 may beassociated with a merchant system associated with the merchant beacondevice 120 and the merchant camera device 140. In yet another exampleembodiment, two payment applications 133 exist, one associated with themerchant system and another associated with the payment processingsystem 160.

In an example embodiment, the user interface 135 enables the merchantPOS device operator 102 to interact with the merchant POS device 130.For example, the user interface 135 may be a touch screen, a voice-basedinterface, or any other interface that allows the merchant POS deviceoperator 102 to provide input and receive output from an application ormodule on the merchant POS device 130. In an example embodiment, themerchant POS device operator 102 interacts via the user interface 135with the payment application 133.

In an example embodiment, the data storage unit 136 comprises a local orremote data storage structure accessible to the merchant POS device 130suitable for storing information. In an example embodiment, the datastorage unit 136 stores encrypted information, such as HTML5 localstorage.

In an example embodiment, the communication application 139, such as aweb browser application or a stand-alone application, enables anoperator of the merchant POS device 130 to view, download, upload, orotherwise access documents or web pages via a distributed network 120.For example, the communication application 139 may enable communicationover the network 120 with the payment processing system 160.

An example merchant camera device 140 comprises a processor 143, a datastorage unit 146, a camera module 147, and a communication application149.

In an example embodiment, the processor 143 performs one or morefunctions described herein as being performed by the merchant cameradevice 140.

In an example embodiment, the data storage unit 146 comprises a local orremote data storage structure accessible to the merchant camera device140 suitable for storing information. In an example embodiment, the datastorage unit 146 stores encrypted information, such as HTML5 localstorage.

In an example embodiment, the camera module 147 may be any module orfunction of the merchant camera device 140 that captures a video inputor captures a digital image of an external environment of the merchantcamera device 140. The camera module 147 may be resident on the merchantcamera device 140 or in any manner logically connected to the merchantcamera device 140. For example, the camera module 147 may be connectedto the merchant camera device 140 via the network 120. The camera module147 may be capable of obtaining individual images or a video scan. Anyother suitable image capturing device may be represented by the cameramodule 147.

In an example embodiment, the communication application 149 enables themerchant camera device 140 to communicate with a server of the paymentprocessing system 160.

An example issuer system 150 approves or denies a payment authorizationrequest received from the payment processing system 160. In an exampleembodiment, the issuer system 150 communicates with the paymentprocessing system 160 over the network 120. In an example embodiment,the issuer system 150 communicates with an acquirer system to approve acredit authorization and to make payment to the payment processingsystem 160 and/or merchant system. For example, the acquirer system is athird party payment processing company.

An example payment processing system 160 comprises an account managementmodule 161, a facial recognition module 163, an audio recognition module165, a data storage unit 166, and a transaction processing module 167.

In an example embodiment, the account management module 161 manages oneor more user 101 accounts. In an example embodiment, a user 101 accountmay comprise a digital wallet account, an email account, a socialnetworking account, or any other appropriate account associated with thepayment processing system 160. In an example embodiment, the accountmanagement system 161 communicates with a payment application 113operating on a user computing device 110 associated with a user 101having a user 101 account with the payment processing system 160. In anexample embodiment, the user 101 enters payment account information intothe user 101 account via the payment application 113 and the accountmanagement module 161 receives the payment account information over thenetwork 120 and associates the received payment account information withthe user 101 account.

In an example embodiment, the facial recognition module 163 receives afacial image of a user 101 associated with a user 101 account submittedby the user 101 via the user computing device 110 over the network 120.For example, the user 101 submits the facial image at the time the user101 establishes the user 101 account with the payment processing system160. In an example embodiment, the facial recognition module 163generates a facial template based on a received facial image.

In an example embodiment, the audio recognition module 165 receives aaudio recording of a user 101 associated with a user 101 accountsubmitted by the user 101 via the user computing device 110 over thenetwork 120. For example, the user 101 submits the audio recording atthe time the user 101 establishes the user 101 account with the paymentprocessing system 160. In an example embodiment, the audio recognitionmodule 165 generates an audio template based on the received audiorecording of the user 101. In an example embodiment, the audiorecognition module 165 receives a audio recording of a first user 101and compares the audio recording of the first user 101 against a storedaudio template associated with a second user 101.

In an example embodiment, the data storage unit 166 comprises a local orremote data storage structure accessible to the payment processingsystem 160 suitable for storing information. In an example embodiment,the data storage unit 166 stores encrypted information, such as HTML5local storage.

In an example embodiment, the transaction processing module 167 receivestransaction details from a merchant POS device 130 and a request toinitiate a transaction. Example transaction details comprise merchantsystem account information, a total amount of the transaction, and auser 101 selection of a user 101 payment account associated with theuser's 101 account with the payment processing system 160. For example,the user's 101 account is a digital wallet account comprising one ormore payment account information corresponding to one or more respectivepayment accounts of the user 101. In an example embodiment, thetransaction processing module 167 extracts payment account informationfrom the user 101 account corresponding to the user 101 selection of theuser 101 payment account received in the transaction details from themerchant POS device 130. In an example embodiment, the transactionprocessing module 167 transmits a payment authorization request to anissuer system 150 or other appropriate financial institution associatedwith the payment account selected by the user 101 for use in thetransaction. An example payment authorization request may comprisemerchant system payment account information, user 101 payment accountinformation, and a total amount of the transaction. In an exampleembodiment, after the issuer system 150 processes the paymentauthorization request, the transaction processing module 167 receives anapproval or denial of the payment authorization request from the issuersystem 150 over the network 120. In an example embodiment, thetransaction processing module 167 transmits a receipt to the merchantPOS device 130 and/or the user computing device 110 comprising a summaryof the transaction.

It will be appreciated that the network connections shown are exampleand other means of establishing a communications link between thecomputers and devices can be used. Moreover, those having ordinary skillin the art having the benefit of the present disclosure will appreciatethat the user computing device 110, the merchant beacon device 120, themerchant point of sale device 130, the merchant camera device 140, theissuer system 150, and the payment processing system 160 illustrated inFIG. 1 can have any of several other suitable computer systemconfigurations. For example, a user computing device 110 embodied as amobile phone or handheld computer may or may not include all thecomponents described above.

In example embodiments, the network computing devices and any othercomputing machines associated with the technology presented herein maybe any type of computing machine such as, but not limited to, thosediscussed in more detail with respect to FIG. 15. Furthermore, anymodules associated with any of these computing machines, such as modulesdescribed herein or any other modules (scripts, web content, software,firmware, or hardware) associated with the technology presented hereinmay by any of the modules discussed in more detail with respect to FIG.15. The computing machines discussed herein may communicate with oneanother as well as other computer machines or communication systems overone or more networks, such as network 120. The network 120 may includeany type of data or communications network, including any of the networktechnology discussed with respect to FIG. 15.

Example Processes

The example methods illustrated in FIGS. 2-14 are described hereinafterwith respect to the components of the example operating environment 100.The example methods of FIGS. 2-7 may also be performed with othersystems and in other environments.

FIG. 2 is a block diagram depicting a method 200 for conducting ahands-free transaction with facial recognition of a user 101, inaccordance with certain example embodiments. The method 200 is describedwith reference to the components illustrated in FIG. 1.

In block 210, the merchant system registers with the payment processingsystem 160 and installs hardware in a merchant location. The method forregistering, by a merchant system, with a payment processing system 160and installing hardware at a merchant system location is described inmore detail hereinafter with reference to the method described in FIG.3.

FIG. 3 is a block diagram depicting a method 210 for registering, by amerchant system, with a payment processing system 160 and installinghardware at a merchant system location, in accordance with certainexample embodiments. The method 210 is described with reference to thecomponents illustrated in FIG. 1.

In the example embodiments described herein, the merchant system doesnot need to install hardware at the example merchant system location inany particular order. The method 210 describes one example method ofinstalling hardware at the merchant location. However, the merchantsystem or other system installing the merchant hardware does not need toinstall the merchant POS device 130, the merchant camera device 140, orthe merchant beacon device 120 in the order described herein.

In block 310, a merchant system registers with the payment processingsystem 160. In an example embodiment, an agent of the merchant systemaccesses a payment processing system 160 website and registers for amerchant account with the payment processing system 160 via the website.In an example embodiment, the merchant system adds payment accountinformation associated with a merchant account to the merchant accountmanaged by the payment processing system 160. In an example embodiment,the merchant system comprises one or more merchant system locations. Forexample, the merchant system may comprise one or more physical storelocations. An example merchant location comprises one or more merchantpoint of sale (“POS”) devices 130. In an example embodiment, one or moremerchant POS device operators 102 operate the one or more merchant POSdevices 130 at the merchant system location.

In block 320, a merchant system operator installs the paymentapplication 133 on the merchant point of sale device 130. In anotherexample embodiment, the merchant system operator purchases a merchantPOS device 130 from the payment processing system 160 with the paymentapplication 133 pre-installed on the merchant POS device 130. In anexample embodiment, the merchant POS device 130 is able to communicatewith the payment processing system 160 over a network 120. In an exampleembodiment, the merchant POS device 130 communicates with the paymentprocessing system 160 via the payment application 133. For example, themerchant POS device 130 may be able to transmit transaction details tothe payment processing system 160 via the payment application 133 overthe network 120 to enable the payment processing system 160 to process atransaction. In another example, the merchant POS device 130 may be ableto receive a receipt from the payment processing system 160 thatnotifies a merchant POS device operator 102 as to whether a transactionwas successful or not.

In block 330, the merchant beacon device 120 receives a beaconidentifier code from the payment processing system 160. In an exampleembodiment, the merchant system receives a beacon identifier from thepayment processing system 160 and installs or otherwise saves the beaconidentifier on the merchant beacon device 120. In an example embodiment,a merchant system operator installs the merchant beacon device 120 inproximity to a merchant POS device 130. In an example embodiment, themerchant system operator installs a plurality of merchant beacon devices120, each merchant beacon device 120 in proximity to one or moreassociated merchant POS devices 130. In an example embodiment, themerchant beacon device 120 is able to broadcast a merchant beaconidentifier over a wireless medium, wherein one or more user computingdevices 110 located within a threshold proximity to the merchant beacondevice 120 are able to receive the merchant beacon identifier over thewireless medium. In another example embodiment, the merchant beacondevice 120 is able to establish a local network 120 connection to one ormore user computing devices 110 located within a threshold proximity tothe merchant beacon device 120 and the merchant beacon device 120transmits the merchant beacon identifier to the one or more usercomputing devices 110 over the established local network 120 connection.For example, the threshold proximity depends on the network 120communication protocol utilized by the merchant beacon device 120.

In block 340, the merchant beacon device 120 broadcasts the beaconidentifier code via wireless communication at the location of themerchant system. For example, the merchant beacon device 120 maybroadcast, emit, or otherwise transmit data comprising the beaconidentifier via Wi-Fi, Bluetooth, Bluetooth low energy (“BLE”), nearfield communication (“NFC”), or other appropriate communication protocolto one or more user computing devices 110 located at the merchant systemlocation within a threshold proximity to the merchant beacon device 120.In some example embodiments, the merchant beacon device 120, at a timebefore transmitting the merchant beacon identifier, is operable toestablish a network 120 connection between the merchant beacon device120 and one or more user computing devices 110 located at the merchantsystem location within a threshold proximity to the merchant beacondevice 120.

In block 350, a merchant system operator installs the merchant cameradevice 140 at the merchant system location to correspond to the merchantbeacon device 120. In an example embodiment, both a merchant cameradevice 140 and a merchant beacon device 120 are installed in proximityto a particular merchant POS device 130. In another example embodiment,a merchant camera device 140 and a merchant beacon device 120 areinstalled in proximity to two or more particular merchant POS devices130. In an example embodiment, the merchant camera device 140 isoriented to be able to capture video and/or images of a face of a user101 standing in front of one or more merchant POS devices 130 during theprocess of checkout. In an example embodiment, the merchant systeminstalls a merchant camera device 140 that is oriented to capture videoand/or images of the face of a user standing in front of a particularmerchant POS device 130. In another example embodiment, the merchantsystem installs a merchant camera device 140 that is oriented to capturevideo and/or images of the faces of one or more users 101 standingwithin a proximity of a particular plurality of merchant POS devices 130within a range of a field of vision of the camera module 147 of themerchant camera device 140.

In block 360, the payment processing system 160 receives a merchantcamera device 140 identifier and associates it with the correspondingbeacon identifier code of the merchant beacon device 120. In an exampleembodiment, the merchant system and/or the payment processing system 160configures the merchant camera device 140 so that the merchant cameradevice 140 is able to communicate with the payment processing system 160over the network 120. An example camera device 140 identifier comprisesa hardware identifier, a MAC address, or other useful or relevantidentifier associated with the merchant camera device 140. In an exampleembodiment, the payment processing system 160 comprises a databasecomprising merchant camera device 140 identifiers and associated beaconidentifiers for merchant beacon device 120 identifiers for a particularmerchant system location. In an example embodiment, the merchant cameradevice transmits the merchant beacon device 120 identifier in additionto the merchant camera device 140 identifier to the payment processingsystem 160. In an example embodiment, the merchant camera device 140,during the setup and installation process, may receive the merchantbeacon device 120 identifier over an appropriate wireless communicationchannel from the merchant beacon device 120. In another exampleembodiment, the merchant camera device 140, during the setup andinstallation process, may establish a network 120 connection with themerchant beacon device 120 and receive the merchant beacon device 120identifier over the network 120. In another example embodiment, thepayment processing system 160 receives the merchant camera device 140identifier, extracts one or more merchant beacon device 120 identifiersfrom the database, and associates the merchant camera device 140identifier with one or more of the one or more extracted merchant beacondevice 120 identifiers. In yet another example embodiment, the merchantsystem operator installs the one or more merchant beacon devices 120after installing the one or more merchant camera devices 140. In thisexample embodiment, the payment processing system 160 generates amerchant beacon device identifier to associate with a merchant cameradevice 140 identifier and transmits the generated merchant beacon deviceidentifier to the merchant system. In this example embodiment, themerchant system operator manually configures the merchant beacon device120 to broadcast, emit, or otherwise transmit the merchant beacon deviceidentifier assigned by the payment processing system 160 over a network120.

In certain example embodiments, one or both of the merchant cameradevice 140 and the merchant beacon device 120 are components of themerchant POS device 130 or are wirelessly or physically connected to themerchant POS device 130 and controlled by one or more processors of themerchant POS device 130. In certain example embodiments, certainfunctions described herein as performed by the merchant camera device140 and/or the merchant beacon device 120 may also be performed by themerchant POS device 130.

From block 360, the method 210 proceeds to block 220 of FIG. 2.

Returning to FIG. 2, in block 220, the user 101 registers with thepayment processing system 160. The method for registering, by a user101, for an account with a payment processing system 160 is described inmore detail hereinafter with reference to the method 220 described inFIG. 4.

FIG. 4 is a block diagram depicting a method 220 for registering, by auser 101, for an account with a payment processing system 160, inaccordance with certain example embodiments. The method 220 is describedwith reference to the components illustrated in FIG. 1.

In block 410, the user 101 accesses the payment processing systemwebsite 169. For example, the user 101 accesses the payment processingsystem 160 via the web browser 118 of the user computing device 110. Inanother example, the user 101 may otherwise contact the paymentprocessing system 160 to register for a user 101 account.

In block 420, the user 101 registers with the payment processing system160. The user 101 may obtain a user account number, receive theappropriate applications and software to install on the user computingdevice 110, request authorization to participate in hands-free paymentprocessing, or perform any action required by the payment processingsystem 160. The user 101 may utilize the functions of the user computingdevice 110, such as the user interface 115 and the web browser 118, toregister and configure a user 101 account. In an example embodiment, theuser 101 may enter payment account information associated with one ormore user 101 accounts, for example, one or more credit accounts, one ormore bank accounts, one or more stored value accounts, and/or otherappropriate accounts into the user 101 account maintained by the paymentprocessing system 160.

In block 430, the user 101 downloads the payment application 113 ontothe user computing device 110. In an example embodiment, the paymentapplication 113 operating on the user computing device 110 is able tocommunicate with the payment processing system 160 over the network 120.In an example embodiment, the user 101 may configure user 101 accountsettings or add, delete, or edit payment account information via thepayment application 113. In an example embodiment, the user 101 mayselect an option to enable or disable the permission of the paymentprocessing system 160 to process hands free transactions. For example, ahands free transaction comprises a transaction wherein the user 101 doesnot need to interact with the user computing device 110 or requiresminimal user 101 interaction with the user computing device 110 toinitiate a transaction with the merchant system.

In block 440, the payment processing system 160 establishes a facialtemplate associated with the user 101 account. The method forestablishing a facial template associated with a user 101 account isdescribed in more detail hereinafter with reference to the method 440described in FIG. 5.

FIG. 5 is a block diagram depicting a method 440 for establishing afacial template associated with a user 101 account, in accordance withcertain example embodiments. The method 440 is described with referenceto the components illustrated in FIG. 1.

In block 510, the payment application 113 displays a request for theuser 101 to capture a facial image via the user computing device 110. Inan example embodiment, the payment application 113 displays the requestvia the user interface 115. In an example, the user interface 115 maydisplay a request that reads, “to enable hands free transactions, weneed an image of your face. Would you like submit a facial image now?”In this example, the user 101 may select an option to take a currentpicture or may otherwise select a picture stored on the user computingdevice 110.

In block 520, the user 101 selects an option to capture a facial image.For example, the user 101 actuates an object on the user interface 115that reads, “yes, I would like to take a picture now.”

In block 530, the payment application 113 activates the camera module117 on the user computing device 110 and the user 101 captures a facialimage of himself. In an example embodiment, the user computing deviceuser interface 115 may display a live camera feed of the user 101 to aidthe user 101 in aligning the user's 101 face to take the facial image.In an example embodiment, the payment application 113 may display on theuser computing device 110 a box or other perimeter on the user interface115 within which the user 101 should align his face to take a picture ofa required size predetermined by the payment processing system 160. Inan example embodiment, the user 101 may actuate an object on the userinterface 115 to capture the image. In this example embodiment, inresponse to the user actuating the object on the user interface 115, thecamera module 117 receives a command from the payment application 113 tocapture an image of the user 101. In another example embodiment, thecamera module 117 receives a command from the payment application 113 tocapture a plurality of images of the user 101 as the user 101 moves thecamera around the user's 101 face. For example, each of the plurality ofimages of the user 101 may correspond to a particular pose of the user's101 face. An example facial image may comprise a digital image of theface of a user 101. In an example embodiment, the payment processingsystem 160 may establish guidelines for users 101 in submitting facialimages. For example, the payment application 113 may direct the user 101to remove any hats, head coverings, glasses, or other objects oraccessories that may occlude regions of the user's 101 face so thatpayment application 160 may receive a complete depiction of the user's101 face.

In an example embodiment, the user computing device 110 determines ifthe captured facial image is a valid facial image or an invalid facialimage. For example, a valid facial image complies with guidelinespredetermined by the payment processing system 160 and an invalid facialimage does not comply with one or more of the guidelines. For example,if the user computing device 110 captures a facial image that comprisesincorrect dimensions, if part or all of the user's 101 face is occluded,or if the image is too dark or too bright, the user computing device 110rejects the invalid facial image and displays a request directing theuser 101 to capture a subsequent facial image. In this example, the user101 captures a subsequent facial image via the user computing device110, and the user computing device 110 transmits the subsequent facialimage to the payment processing system 160 via the network 120.

In block 540, the payment processing system 160 receives the facialimage. In another example embodiment, the payment processing system 160receives a plurality of facial images of the user 101. For example, thepayment application 113 transmits the one or more facial images of theuser 101 to the payment processing system 160 via the network 120. In anexample embodiment, the payment processing system 160 associates thereceived one or more facial images with the user 101 account. Forexample, the payment processing system 160 is able to identify the user101 account to associate with the received one or more images becausethe user 101 is currently logged in to the payment application 113 onthe user computing device 110 at the time the one or more facial imagesare transmitted to the payment processing system 160. In certain exampleembodiments, the payment processing system 160 determines if thereceived facial image is a valid facial image or an invalid facialimage. For example, a valid facial image complies with all guidelinespredetermined by the payment processing system 160 and an invalid facialimage does not comply with one or more of the guidelines. For example,if a user 101 submits a facial image that comprises incorrectdimensions, if part or all of the user's 101 face is occluded, or if theimage is too dark or too bright, the payment processing system 160rejects the invalid facial image and transmits a request to the usercomputing device 110 directing the user 101 to capture a subsequentfacial image to transmit to the payment processing system 160. In thisexample, the user computing device 110 receives and displays therequest, the user 101 captures a subsequent facial image via the usercomputing device 110, and the user computing device 110 transmits thesubsequent facial image to the payment processing system 160 via thenetwork 120.

In block 550, the payment processing system 160 creates a facialtemplate associated with the user 101 account based on the receivedfacial image. In another example embodiment, the payment processingsystem 160 generates a corresponding facial template for each of aplurality of received facial images associated with the user 101account. In an example, the facial template is of a predetermined size,for example, a 128-byte facial template. In an example, the paymentprocessing system 160 generates a facial template comprising a computercode representation of the digital facial image. For example, the facialtemplate may describe key features of the facial image of the user 101,such as shape, color, line, value, space, form, texture, or other usefulor relevant feature of the image or of particular regions of the image.In an example, the facial template is generated by processing the facialimage through a convolutional neural network. In an example embodiment,the payment processing system 160 stores the generated facial templateassociated with the user 101 in a data storage unit 166 associated withthe payment processing system 160. For example, the payment processingsystem 160 database may comprise a table or other means by which itcorrelates each user 101 account identifier with an associated facialtemplate of the user 101.

In another example embodiment, after the user computing device 110captures one or more facial images of the user 101, the user computingdevice 110 generates one or more facial templates corresponding to oneor more of the one or more captured facial images of the user 101. Inthis example embodiment, the user computing device 110 transmits the oneor more generated facial templates to the payment processing system 160over the network 120.

In block 560, the payment processing system 160 deletes the receivedfacial image. For example, the payment processing system 160 only uses afacial template comprising a computer code representation of the facialimage of the user 101. In another example, the payment processing system160 saves the received facial image for future processing. For example,the payment processing system 160, at a later time, updates a facialtemplate generation algorithm and generates an updated facial templatecorresponding to the saved facial image.

From block 560, the method 440 proceeds to block 450 in FIG. 4.

Returning to block 450, in FIG. 4, the payment processing system 160establishes a audio template associated with the user 101 account. Themethod for establishing an audio template associated with a user 101account is described in more detail hereinafter with reference to themethod 450 described in FIG. 6.

FIG. 6 is a block diagram depicting a method 450 for establishing anaudio template associated with a user 101 account, in accordance withcertain example embodiments. The method 450 is described with referenceto the components illustrated in FIG. 1.

In block 610, the payment application 113 displays a request for theuser 101 to capture an audio recording of the user's 101 voice via theuser computing device 110. In an example embodiment, the paymentapplication 113 displays the request via the user interface 115. In anexample, the user interface 115 may display a request that reads, “toenable hands free transactions, we need recording of your voice. Wouldyou like submit a audio recording now?” In this example, the user 101may select an option to submit a live audio recording or may otherwiseselect a pre-recorded audio recording of the user 101 stored on the usercomputing device 110.

In block 620, the user 101 selects an option to capture an audiorecording. For example, the user 101 actuates an object on the userinterface 115 that reads, “yes, I would like to submit a audio recordingnow.”

In block 630, the payment application 113 activates an audio module (notdepicted) on the user computing device 110 and the user 101 captures anaudio of the user's 101 voice. In an example embodiment, the usercomputing device user interface 115 may display guidelines received fromthe payment processing system 160 to aid the user 101 in submitting aaudio recording. For example, the payment application 113 may displaydirections for the user 101 to record the user's 101 voice saying theuser's 101 name. In another example, the payment application 113 directsthe user 101 to capture a audio recording to establish a voice passwordthat can be used in hands-free transactions of the user 101. In anexample embodiment, the user 101 may actuate an object on the userinterface 115 to capture or record a live audio recording. In thisexample embodiment, in response to the user actuating the object on theuser interface 115, the audio module receives a command from the paymentapplication 113 to capture a audio recording of the user 101. In anexample embodiment, the payment processing system 160 may establishguidelines for users 101 in submitting audio recordings. For example,the payment application 113 may direct the user 101 to submit a audiorecording no longer than a predefined maximum length of duration. Forexample, the audio recording must be no longer than five seconds. Inanother example, the payment application 113 may direct the user 101submit a audio recording lasting a predefined length of time, forexample, a audio recording lasting five seconds. In an exampleembodiment, after the user 101 initiates a audio recording by actuatingan object of the user interface 115, the payment application 113transmits a command to the audio module to stop recording the user's 101voice after the predefined length of time or predefined maximum lengthof time has expired.

In block 640, the payment processing system 160 receives an audiorecording of the voice of the user 101. For example, the paymentapplication 113 or audio module transmits the audio recording to thepayment processing system 160 over the network 120. In another example,the payment application 113 retrieves an audio recording selected by theuser 101 and saved in the data storage unit 116 and transmits theretrieved audio recording to the payment processing system 160 over thenetwork 120.

In block 650, the payment processing system 160 creates an audiotemplate associated with the user's 101 voice based on the receivedaudio of the voice of the user. In an example, the audio template is ofa predetermined size. In an example, the payment processing system 160generates a audio template comprising a computer code representation ofthe user's 101 audio recording. For example, the audio template maydescribe key features of the audio recording of the user 101, such asthe intonation of the user's 101 voice or other features of the user's101 voice. [Inventors: does this work similarly to the face template?]In an example embodiment, the payment processing system 160 stores thegenerated audio template associated with the user 101 in a data storageunit 166 associated with the payment processing system 160. For example,the payment processing system 160 database may comprise a table or othermeans by which it correlates each user 101 account identifier with anassociated audio template of the user 101

In block 660, the payment processing system 160 deletes the receivedaudio of the voice of the user 101. In an example embodiment, thepayment processing system 160 deletes the received audio recording ofthe user 101 to protect the privacy of the user 101. For example, thepayment processing system 160 only uses a audio template comprising acomputer code representation of the audio recording of the user 101.

From block 660, the method 440 proceeds to block 230 in FIG. 2.

Returning to block 230, in FIG. 2, the user 101 enters the merchantsystem location and signs into the payment application 113 on the usercomputing device 110. In another example embodiment the user 101 signsinto the payment application 113 at a time before entering the merchantsystem location and enters the merchant location carrying the usercomputing device 110 signed into the payment application 113.

In block 240, the user device receives a merchant beacon device 120identifier. The method for receiving, by a user computing device 110, amerchant beacon identifier broadcast by a merchant beacon device 120 isdescribed in more detail hereinafter with reference to the method 240described in FIG. 7.

FIG. 7 is a block diagram depicting a method 240 for receiving, by auser computing device 110, a merchant beacon identifier broadcast by amerchant beacon device 120, in accordance with certain exampleembodiments. The method 240 is described with reference to thecomponents illustrated in FIG. 1.

In block 710, the user 101 signs in to the payment application 113 onthe user computing device 110. In an example embodiment, the user 101may have a username and password associated with the user 101 accountmaintained by the payment processing system 160. In an exampleembodiment, the user 101 opens the payment application 113 on the usercomputing device 110 and enters a username and/or password via the userinterface 115 to sign in to the payment application 113. In an exampleembodiment, when the user 101 is signed in to the payment application113, the payment application is able to communicate with the paymentprocessing system 160 over the network 120. In this example embodiment,when the user 101 is not signed in to the payment application 113, thepayment application does not communicate with the payment processingsystem 160 even if the a network 120 connection is available. In anexample embodiment, the user 101 may sign out of the payment application113 at any time by actuating one or more objects on the user interface115 of the user computing device 110. In an example embodiment, aftersigning in to the payment application 113, the user 101 configure one ormore user 101 account settings, add, edit, or delete user 101 paymentaccount information, and/or change user 101 preferences. In certainexample embodiments, a user 101 may be required to make a featureselection to obtain the benefits of the techniques described herein. Forexample, the user 101 may have to enable one or more user 101 accountsettings to enable hands free transactions according to the methodsdescribed herein.

In an example embodiment, payment application 113 may provide options,data, configurable alerts, and other suitable features to the user 101.For example, the payment application 113 may comprise a listing ofmerchant systems and merchant locations that participate in hands freepayment transactions according to one or more of the methods describedherein. The listing may be updated periodically from the paymentprocessing system 160. The payment application 113 may notify the user101 when the user 101 is within a configured vicinity of a participatingmerchant system. The payment application 113 may provide the user 101with options to update payment preferences. The payment application 113may provide the user 101 with a listing of recent transactions. Thepayment application 113 may provide any other suitable information tothe user 101.

In block 720, the user 101 carries the user computing device 110 withina threshold distance of the merchant beacon device 120 at the merchantsystem location. In an example embodiment, the user 101 enters alocation of the merchant system. The user 101 may enter the merchantlocation carrying the user computing device 110 in a pocket or a bag, inthe hands of the user 101, or in any suitable manner. The location ofthe merchant system may be a store location, a kiosk location, or anysuitable physical location of a merchant system. In another exampleembodiment, a merchant POS operator 102 may be mobile and arrive at alocation of the user 101. For example, the merchant system may be arestaurant and the merchant POS device operator 102 may be a deliveryperson possessing a portable merchant POS device 130.

In certain example embodiments, the payment application 113 may alertthe user 101 when the user 101 is in the vicinity of a merchant systemthat accepts hands-free payments. The alert may be provided via amessage on the user computing device 110, via an email or a text, or inany suitable manner. In an example embodiment, the alert may be based onthe location of the user 101 as determined by a GPS module (notdepicted) resident on the user computing device 110. For example, thepayment application 113 accesses the GPS data from the GPS module andcompare the GPS location to a list of locations of merchant systems thataccept hands free payments. For example, the payment application 113comprises a list or accesses a list maintained by the payment processingsystem 160 of merchant system locations that accept hands free payments.If a match results from the comparison, then an alert is generated andprovided to the user 101. The match may result if the user 101 is withina configured distance of a qualified merchant system location. In anexample embodiment, the alerts may be configured to alert in anysuitable manner. In an example, the alerts may be combined incommercially dense environments or the alerts may be presentedindividually. In another example, the alerts may be configured to onlyalert the user 101 a configured number of times. For example, the alertmay be presented three times, but upon a fourth instance, the alert isnot presented. The alerts may be presented as a notification with anaudible alert, a vibration, a popup alert on the user interface 115 ofthe user computing device 110, or other suitable alert.

In block 730, the user computing device 110 receives a merchant beaconidentifier broadcast by the merchant beacon device 120. The usercomputing device 110 recognizes a merchant beacon device 120 viawireless communication at the location of the merchant system. The usercomputing device 110 may be configured to search for beacons or otherwireless signals. In an example embodiment, the user computing device110 and the merchant beacon device 120 establish a Wi-Fi wirelessnetwork 120 connection. In other example embodiments, the user computingdevice 110 and the merchant beacon device 120 establish a Bluetooth,BLE, NFC, or other appropriate network 120 connection. Upon entering therange of the signal of the merchant beacon device 120, the usercomputing device 110 receives the merchant beacon identifier.

In block 740, the user computing device 110 transmits the receivedmerchant beacon identifier and a user 101 account identifier to thepayment processing system 160. In an example embodiment, the usercomputing device 110 transmits the data received in the merchant beaconidentifier along with a user 101 account identifier to the paymentprocessing system 160 over the network 120.

In block 750, the payment processing system 160 receives the merchantbeacon identifier and the user 101 account identifier. For example, thepayment processing system 160 receives the merchant beacon identifierand the user 101 account identifier over the network 120. The usercomputing device 110 may compare the data from the merchant beaconidentifier to a database of merchant beacon identifier data and merchantcamera device identifier data to determine an identity of the merchantsystem and merchant camera device 140 associated with the merchantbeacon identifier and/or to verify the authenticity of the beacon.

From block 750, the method 240 proceeds to block 250 in FIG. 2.

Returning to FIG. 2, in block 250, the merchant camera device 140receives a facial template for each user 101 in range of the merchantbeacon device 120. The method for receiving, by a merchant camera device140, a facial template for each user 101 in range of the merchant beacondevice 120 is described in more detail hereinafter with reference to themethod 250 described in FIG. 8.

FIG. 8 is a block diagram depicting a method 250 for receiving, by amerchant camera device 140, a facial template for each user 101 in rangeof the merchant beacon device 120, in accordance with certain exampleembodiments. The method 250 is described with reference to thecomponents illustrated in FIG. 1.

In block 810, the payment processing system 160 extracts a facialtemplate associated with the user 101 account identifier. In an exampleembodiment, the payment processing system 160 accesses a databasecomprising stored facial templates of a plurality of users 101 withcorresponding user 101 account identifiers for each user 101. Forexample, this database is stored in the data storage unit 166.

In block 820, the payment processing system 160 identifies a merchantcamera device 120 associated with the merchant beacon device 120identifier. In an example embodiment, the payment processing system 160recognizes that the merchant beacon identifier is associated with thepayment processing system 160 and a particular merchant camera device140 at the merchant system location. In an example embodiment, thepayment processing system 160 recognizes that the merchant beaconidentifier is associated with a plurality of merchant camera devices 140installed at a particular merchant location.

In block 830, the payment processing system 160 transmits the facialtemplate of the identified user 101 to the merchant camera device 140associated with the merchant beacon device 120 identifier. In anotherexample embodiment, the payment processing system 160 transmits thefacial template of the identified user 101 to a plurality of merchantcamera devices 140 associated with the merchant beacon device 120identifier. In certain example embodiments, the payment processingsystem 160 receives, in real time, a plurality of transmissions fromuser computing devices 101 corresponding to a plurality of users 101present at the merchant system location, each transmission comprising auser 101 account identifier and a retransmitted merchant beaconidentifier. In these example embodiments, the payment processing system160 retrieves, in response to receiving each such transmission, a facialtemplate associated with the received user 101 account identifier andtransmits a facial template to one or more merchant camera devices 140at the merchant location associated with the merchant beacon identifier.

In block 840, the merchant camera device 140 receives the facialtemplate of the user 101. In another example embodiment, a plurality ofmerchant camera devices 140 receive the facial template of the user 101.In yet another example embodiment, the merchant camera device 140 and/orthe plurality of merchant camera devices 140 receive one or moreadditional facial templates from the payment processing system 160corresponding to one or more users 101 other than the instant user 101having user computing devices 110 in network 120 connection to amerchant beacon device according to the method previously describedherein. For example, the one or more additional facial templates arereceived in real time from the payment processing system 160 asadditional users 101 other than the instant user 101 receive themerchant beacon device 120 identifier over a wireless communicationnetwork 120 or otherwise establish a network 120 connection betweentheir user computing devices 110 and one or more merchant beacon devices120. For example, the one or more merchant camera devices 140 mayreceive one or more additional facial templates corresponding to one ormore additional users 101 at a time before, at the same time, or afterthe time at which the merchant camera device 140 receives the facialtemplate of the instant user 101.

In block 850, the merchant camera device 140 adds the facial template ofthe user 101 to a current customer log. In an example embodiment, thecurrent customer log is accessible by the merchant camera device 140 andby the payment processing system 160. In another example embodiment, thecurrent customer log is additionally accessible by the merchant POSdevice 130 and/or an applicable computing device of the merchant system.In an example embodiment, the payment processing system 160 comprisesand maintains the current customer log, which the merchant camera device140 may access via the network 120. In another example embodiment, themerchant camera device 140 comprises and maintains the current customerlog, which is accessible to the payment processing system 160 bycommunicating with the user computing device 110 via the network 120.

In block 860, the merchant camera device 140 periodically updates thecurrent customer log. The method for receiving, by a merchant cameradevice 140, notification from a payment processing system 160 as users101 enter or leave a network range of a merchant beacon device 120 isdescribed in more detail hereinafter with reference to the method 860described in FIG. 9.

FIG. 9 is a block diagram depicting a method 860 for receiving, by amerchant camera device 140, notification from a payment processingsystem 160 as users 101 enter or leave a network range of a merchantbeacon device 120, in accordance with certain example embodiments. Themethod 860 is described with reference to the components illustrated inFIG. 1.

In block 910, the merchant camera device 140 is notified by the paymentprocessing system 160 as users 101 signed into a payment account enteror leave a network range of the merchant beacon device 120. For example,as previously discussed, when a user 101 carrying a user computingdevice 110 enters a threshold distance from a merchant beacon device120, the merchant beacon device 120 or the user computing device 110 ofthe user 101 are able to detect the other device and establish awireless network 120 connection between the two devices at the merchantsystem location. In this example, the merchant beacon device 120transmits the merchant beacon identifier corresponding to the merchantbeacon device 120 over the wireless network 120 to the user computingdevice 110. For example, the merchant beacon device 120 transmits themerchant beacon identifier to the user computing device 110 via a Wi-Fi,Bluetooth, BLE, or NFC wireless communication network 120. In thisexample, the user computing device 110 retransmits the received merchantbeacon identifier to the payment processing system 160 along with a user101 account identifier identifying the user 101.

In block 920, the merchant camera device 140 determines whether a newuser 101 is in range of the merchant beacon device 120. For example, ifthe payment processing system 160 receives a new user 101 accountidentifier in addition to the same merchant beacon identifier, thepayment processing system 160 may determine that a new user 101 is inrange of the merchant beacon device 120. In this example, the paymentprocessing system 160 may infer that the new user 101 has entered themerchant location based on receipt of the new user 101 accountidentifier. In another example, if the payment processing system 160does not receive any new user 101 account identifiers along with thesame merchant beacon identifier within a threshold length of time, thepayment processing system 160 may determine that no new users 101 haveentered the network 120 range of the merchant beacon device 120.

If a new user 101 is in range of the merchant beacon device 120, themethod 860 proceeds to block 830 in FIG. 8. For example, the paymentprocessing system 160 receives a new user 101 account identifier inaddition to the same merchant beacon identifier. In this example, thepayment processing system 160 infers that the new user 101 has enteredthe merchant location based on receipt of the new user 101 accountidentifier and the same merchant beacon identifier as previouslyreceived from the first user 101.

Returning to FIG. 8, in block 810, the payment processing system 160extracts a facial template associated with the new user 101 accountidentifier. In an example embodiment, the payment processing system 160transmits the facial template to the appropriate one or more merchantcamera devices 140 and the one or more merchant camera devices 140 addthe new user's 101 facial template to the current customer log accordingto the example method previously described in method 250 in FIG. 8.

Returning to FIG. 9, in block 920, if there is a not a new user 101 inrange of the merchant beacon device 120, the method 860 proceeds toblock 930. For example, the payment processing system 160 does notreceive any new user 101 account identifiers along with the samemerchant beacon identifier within a threshold length of time anddetermines that no new users 101 have entered the network 120 range ofthe merchant beacon device 120.

In block 930, the merchant camera device 140 determines whether acurrent user 101 has moved out of range of the merchant beacon device120. In an example embodiment, the user computing device 110 continuesto receive from the merchant beacon identifier from the merchant beacondevice 120 and retransmit the merchant beacon identifier along with theuser 101 account identifier to the payment processing system 160. Inthis example embodiment, the user computing device 110 may periodicallytransmit information comprising the merchant beacon identifier and user101 account identifier to the payment processing system 160 as long asthe user computing device 110 continues to detect the merchant beacondevice 120 and receive the merchant beacon device 120 identifier viaperiodic scans. For example, the user computing device scans for themerchant veacon device 120 every five seconds. In another exampleembodiment, the user computing device 110 may periodically transmitinformation comprising the merchant beacon identifier and user 101account identifier to the payment processing system 160 as long as theuser computing device 110 maintains a wireless network 120 connectionwith the merchant beacon device 120. For example, the user computingdevice 110 may transmit this information to the payment processingsystem 160 at every five seconds. In this example embodiment, if thepayment processing system 160 ceases to receive the information from theuser computing device for a predefined number of intervals, the paymentprocessing system 160 may determine that the corresponding user 101 hasmoved out of range of the merchant beacon device. In this exampleembodiment, if the payment processing system 160 continues to receivethe information transmitted by the user computing device 110 at theexpected intervals, the payment processing system 160 determines thatthe user 101 is still in network 120 range of the merchant beacon device120.

If no current user 101 has moved out of range of the merchant beacondevice 120, the method 860 proceeds to block 260 in FIG. 2. For example,the payment processing system 160 continues to receive the merchantbeacon identifier and user 101 account identifier transmitted by theuser computing device 110 at the expected intervals and determines thatthe user 101 is still in network 120 range of the merchant beacon device120.

Returning to block 260, in FIG. 2, the user 101 initiates a transactionat the merchant POS device 130.

Returning to FIG. 9, in block 930, if a current user 101 has moved outof range of the merchant beacon device 120, the method 860 proceeds toblock 940.

In block 940, the merchant camera device 140 receives a notificationfrom the payment processing system 160 that a current user 101 is out ofnetwork range of the merchant beacon device 120. In another exampleembodiment, the merchant camera device 140 receives a notification fromthe payment processing system 160 that the user computing device 110associated with the current user 101 has stopped sending notificationsto the payment processing system 160 comprising the merchant beacondevice 120 identifier. For example, the merchant camera device 140receives the user 101 account identifier associated with the currentuser 101 associated with a user computing device 110 that is either outof network range or has stopped transmitting notifications comprisingthe merchant beacon device 120 identifier to the payment processingsystem 160, accesses the current customer log, and finds an entrycorresponding to the current user 101. For example, the current customerlog is maintained by the payment processing system 160 and the merchantcamera device 140 accesses the current customer log over the network 120by communicating with the payment processing system 160. In anotherexample embodiment, the payment processing system 160 does not transmita notification to the merchant camera device 140 that the current user101 is out of network range. In this example embodiment, the paymentprocessing system 160 accesses the current customer log and deletes thefacial template of the current user 101 from the current customer log.

In block 950, the merchant camera device 140 deletes the facial templateof the current user 101 from the current customer log. For example, thecurrent customer log comprises a table and the merchant camera device140 deletes or requests the deletion of an entry or row corresponding todata associated with the current user 101 for which the merchant cameradevice 140 received the notification. In another example embodiment, thepayment processing system 160 accesses the current customer log anddeletes the facial template of the current user 101 from the currentcustomer log.

From block 950, the method 860 proceeds to block 260, in FIG. 2.

Returning to FIG. 2, in block 260, the user 101 initiates a transactionat the merchant point of sale device 130. The method for initiating, bya user 101, a transaction at a merchant point of sale device 130 isdescribed in more detail hereinafter with reference to the method 260described in FIG. 10. In the example embodiments described herein, theuser 101 initiates a “hands free transaction” at the merchant POS device130. An example hands free transaction does not require any interactionwith the user computing device 110 on the part of the user 101. Inanother example, a hands free transaction requires only minimalinteraction with the user computing device 110 by the user 101.

FIG. 10 is a block diagram depicting a method 260 for initiating, by auser 101, a transaction at a merchant point of sale device 130, inaccordance with certain example embodiments. The method 260 is describedwith reference to the components illustrated in FIG. 1.

In block 1010, the user 101 approaches the merchant point of sale device130. In an example embodiment, at a time prior to approaching themerchant POS device 130, the user 101 browses the merchant systemlocation and selects one or more items to purchase. In this exampleembodiment, the user 101 may collect the one or more items and carry, orotherwise transport via physical basket or shopping cart, the one ormore items to the merchant POS device 130.

In block 1020, the merchant point of sale device 130 operator 102 totalsthe items of the user 101 for purchase. In an example embodiment, themerchant POS device operator 102 scans barcodes attached to the one ormore items or otherwise enters descriptions and prices associated withthe one or more items into the merchant POS device 130. In an exampleembodiment, after scanning or manually entering the items into themerchant POS device 130, the merchant POS device operator 102 actuatesan object on the user interface 135 of the merchant POS device 130 toorder the merchant POS device 130 to total the items. In an exampleembodiment, the merchant POS device 130 displays, via the user interface135, the total to the user 101.

In block 1030, the merchant point of sale device 130 operator asks theuser 101 to select a payment option. In an example embodiment, themerchant POS device 130 displays one or more payment options that theuser 101 may select to use in a transaction. Example payment options maycomprise payment via a payment application 113 associated with thepayment processing system 160, payment by cash, payment by check,payment by credit card, payment by debit card, and/or any other means ofpayment that the merchant system can or is willing to accept for paymentfrom the user 101. In an example embodiment, the one or more paymentoptions are displayed as objects on the user interface 135 and areselectable by the merchant POS device operator 102 in response to theuser 101 directing the merchant POS device 102 operator to make aselection.

In another example embodiment, the payment processing system 160 ormerchant camera device 140 transmits a notification to the merchantpoint of sale device 140 that the user 101 is able to initiate atransaction via the payment application 113. In this example embodiment,the merchant point of sale device 140 notifies the merchant point ofsale device 140 operator that the user 101 is able to participate intransaction using the payment application 113. In this exampleembodiment, the merchant point of sale device 140 operator asks the user101 to confirm whether the user 101 wants to initiate a transactionusing the payment application 113.

In block 1040, the user 101 directs the merchant point of sale deviceoperator 102 to initiate a transaction via the payment application 113.In an example embodiment, in response to receiving a verbal request fromthe user 101 to select the payment application 113 as a payment option,the merchant POS device operator 102 actuates an object on the userinterface 135 of the merchant POS device 130 corresponding to thepayment application 113 payment option.

In block 1050, the merchant point of sale device operator 102 selects anoption on the merchant point of sale device 130 to initiate atransaction using the payment application 113. In an example embodiment,the merchant POS device 130 displays a confirmation screen after themerchant POS device operator 102 selects an option to initiate atransaction using the payment application 113. An example confirmationscreen may display information summarizing the potential transaction andcomprising one or more of a transaction total, a description of the oneor more items being purchased by the user 101, and a indication that theuser 101 selected the payment application 113 as the method of paymentfor the transaction. An example confirmation screen may further displayoptions to confirm the transaction or cancel the transaction. In anexample embodiment, the user 101 reviews the confirmation screen,determines that the information displayed on the confirmation screen iscorrect, determines to continue with the transaction, and directs themerchant POS device operator 102 to select the option to confirm thetransaction via the user interface 135.

In another example embodiment, the user 101 decides to abandon thetransaction because the information is incorrect or because the user 101changed his mind and decided not to purchase the items. In yet anotherexample embodiment, the confirmation screen further comprises an optionto edit the transaction details. In this example embodiment, themerchant POS device operator 102, upon direction of the user 101, mayselect the option to edit the transaction details and may then edit,add, or delete one or more of the items in the transaction or editpayment details or payment methods for the transaction.

In block 1060, the merchant point of sale device 130 transmits a requestto identify the user 101 to the payment processing system 160. In anexample embodiment, the merchant point of sale device transmits arequest to process a transaction along with the request to identify theuser 101.

For example, at this point, the payment processing system 160 is awarethat one or more users 101 are physically present in the merchant systemlocation based on the current customer log maintained by the paymentprocessing system and accessible and editable by the merchant cameradevice 120. Additionally, the payment processing system 160 is awarethat a user 101 has initiated a transaction with the payment application113 payment method based on the request to process the transactionreceived from the merchant POS device 130. However, the paymentprocessing system 160 may not be aware of exactly which user 101requested to process the hands free transaction involving the paymentapplication 113.

In block 1070, the payment processing system 160 receives the request toidentify the user 101 from the merchant point of sale device 130. Forexample, the payment processing system 160 receives the request toidentify the user and the request to process the transaction over thenetwork 120.

In block 1080, the payment processing system 160 transmits a request tothe merchant camera device 140 to identify the user 101. For example,the payment processing system 160 transmits the request to identify theuser to the merchant camera device 140 over the network 120. In anotherexample embodiment, the merchant camera device 140 has alreadycommunicated the identity of the user 101 to the payment processingsystem 160 or to the merchant point of sale device 130 and the paymentprocessing system 160 does not need to transmit a request to themerchant camera device 140 to identify the user 101.

From block 1080, the method 260 proceeds to block 270 in FIG. 2.

Returning to FIG. 2, in block 270, the merchant camera device 140identifies the user 101 via facial recognition. The method foridentifying, by a merchant camera device 140, a user 101 via facialrecognition is described in more detail hereinafter with reference tothe method 270 described in FIG. 11.

FIG. 11 is a block diagram depicting a method 270 for identifying, by amerchant camera device 140, a user 101 via facial recognition, inaccordance with certain example embodiments. The method 270 is describedwith reference to the components illustrated in FIG. 1.

In block 1110, the merchant camera device 140 captures video of the user101. In an example embodiment, in response to receiving a request toidentify the user 101, the merchant camera device 140 activates thecamera module 147 to begin to capture a video of the surroundings of themerchant camera device 140. In an example embodiment, the merchant POSdevice 130 captures a video feed of the user's 101 face. In anotherexample embodiment, the merchant camera device 140 continuouslycaptures, but does not record, a video feed of its surroundings. In thisexample embodiment, when the merchant camera device 140 receives arequest to identify the user 101 from the payment processing system 160,the merchant camera device beings to record the video feed for athreshold amount of time. In an example embodiment, the user 101 may bemoving during the period in which the merchant camera device 140 recordsthe video feed. In an example embodiment, a processor 143 of themerchant camera device 140 extracts a facial image by determining aparticular frame of the video feed and area of the instance of the videofeed corresponding to the face of the user.

In block 1120, the merchant camera device 140 extracts a facial image ofthe user 101 from the captured video. In an example embodiment, themerchant camera device 140 determines a frame of the captured video toprovide an image of the user's 101 face and extracts the frame of thecaptured video comprising the facial image of the user 101.

In certain other example embodiments, the merchant camera device 140determines a frame of the captured video to provide an image of thefaces of a plurality of users 101. For example, the frame comprises animage of the face of a first user 101, a second user 101, and a thirduser 101 at different locations in the image. In this example, onemerchant camera device 140 may capture video of an environmentcorresponding to an area in the proximity of multiple merchant POSdevices 130. In this example embodiment, the merchant camera device 140may determine to which particular merchant POS device 130 each of theplurality of faces of the corresponding plurality of users 101 in theextracted image.

In an example embodiment, in which a merchant camera device 140 ismounted at an angle parallel to a plurality of adjacent merchant POSdevices 130, the field of view of the merchant camera device 140 may bedivided into bounding boxes such that each bounding box is assigned to aparticular merchant POS device 130. In this example embodiment, sinceone or more users 101 may be situated between any two adjacent merchantPOS devices 130, the bounding boxes may overlap. In this exampleembodiment, each bounding box is assigned to at most two adjacentmerchant POS devices 130. In this example embodiment, in which themerchant camera device 140 is mounted parallel to a plurality ofadjacent merchant POS devices 130, the size of a detected facial imageof a user 101 is directly proportional to the distance the user 101 isfrom a merchant POS device 130 detected in the image. In this exampleembodiment, the merchant camera device 140 assigns each detected facialimage to one or two particular merchant POS devices 130 based on theposition of the facial image relative to the bounding boxes.

In another example embodiment, the merchant camera device 140 is mountedat an angle that is not parallel to a plurality of adjacent merchant POSdevices 130. In this example embodiment, the merchant camera device 140may be able to monitor an environment comprising a greater number ofadjacent merchant POS devices 130 than the example embodiment in whichthe merchant camera device 140 is mounted parallel to adjacent merchantPOS devices 130. In this example embodiment, however, the size of adetected facial image is not proportional to the distance between amerchant POS terminal detected in the image and the detected facialimage. In this example embodiment, the merchant camera device 140assigns each detected facial image to one or two particular merchant POSdevices 130 based on the position of the facial image relative to thebounding boxes.

In block 1130, the merchant camera device 140 generates a facialtemplate from the captured facial image. In an example, the facialtemplate is of a predetermined size, for example, a 128-byte facialtemplate. In an example, the payment processing system 160 generates afacial template comprising a computer code representation of the digitalfacial image. For example, the facial template may describe key featuresof the facial image of the user 101, such as shape, color, line, value,space, form, texture, or other useful or relevant feature of the imageor of particular regions of the image. In another example, the facialtemplate is generated by processing the facial image through aconvolutional neural network. In an example embodiment, the merchantcamera device 140 stores the generated facial template in a data storageunit 146 associated with the merchant camera device 140. For example,the merchant camera device 140 database may comprise a log of facialtemplates of current customers wherein the merchant camera device 140stores the generated facial template.

In certain other example embodiments, the merchant camera device 140continuously captures a video feed of its surroundings as users 101enter and leave the vicinity of one or more merchant POS devices 130over the course of a certain time period. In this example embodiment,the merchant camera device 140 processor 147 is able to continuouslymonitor the incoming video feed to detect faces from extracted frames ofthe video feed. In this example embodiment, the merchant camera device140, each time the processor 147 detects the presence of one or morefaces in the video feed, the merchant camera device 140 extracts a frameof the video feed comprising one or more facial images of one or morecorresponding detected faces and creates facial templates based on theextracted one or more facial images. In this example embodiment, themerchant camera device 140 stores facial templates in the log of facialtemplates of current customers as they are generated. In this exampleembodiment, as the merchant camera device 140 generates a subsequentfacial templates, the merchant camera device 140 determines whether thegenerated subsequent facial template is similar to within a thresholdcompared to any of the facial templates already stored in the log offacial templates of current customers. If the generated subsequentfacial template is similar to within a threshold to any of the facialtemplates already stored in the log, the merchant camera device 140,after associating the facial template to one or two particular merchantPOS devices 130 based on the position of the associated facial images inthe extracted frame of the captured video, adds the facial template tothe log of facial templates of current customers. If the generatedsubsequent facial template is not similar to within a threshold to anyfacial templates already stored in the log of facial templates ofcurrent customers, the merchant camera device 140 deletes or otherwiseignores and/or does nothing with the generated facial template. In thisexample embodiment, if the merchant camera device 140 determines thatcertain facial image is no longer in the field of the video feed, thecorresponding facial template is deleted from the log of facialtemplates of current customers.

In block 1140, the merchant camera device 140 deletes the captured videoand the extracted facial image. For example, the merchant camera device140 does not store captured images or video. In this example, facialtemplates generated by the merchant camera device 140 comprise computercode representations of facial images of users 101. In this example,after generating a facial template or after a threshold time has passedafter capturing video or images or extracting an image from a video, themerchant camera device 140 deletes any captured or extracted video orimages.

In block 1150, the merchant camera device 140 retrieves facial templatesfrom the current customer log. For example, the current customer logcomprises facial templates received from the payment processing system160 corresponding to all current users 101 whose associated usercomputing devices 110 are located within a network distance of amerchant beacon device 120.

In block 1160, the merchant camera device 140 compares the generatedfacial template from captured facial image to facial templates from thecurrent customer log.

In block 1170, the merchant camera device 140 determines whether thereis a match between the generated facial template and one of the facialtemplates from the current customer log. In this example embodiment,when the merchant camera device 140 determines a match between a firstfacial template in the log of facial templates of current customers anda second facial template the log of current customers, the merchantcamera device 140 determines with which merchant one or two POS devices130 the second facial template is associated. In this exampleembodiment, the merchant camera device 140 updates the entry of thesecond facial template in the log of current customers with anannotation assigning the second facial template to a particular one ortwo merchant POS devices 130. For example, as previously discussed, thefacial image upon which the facial template was located within abounding box of an area of the video or image corresponding to one ortwo particular merchant POS devices 130. In this example, when themerchant camera device 140 generates a facial template for the facialimage, the merchant camera device 140 associates the generated facialtemplate with the particular one or two merchant POS devices 130. Forexample, if the facial image was in a bounding box in front of a firstmerchant POS device 130, the resulting facial template may be associatedwith the first merchant POS device 130. In another example, if thefacial image was in a bounding box between the first merchant POS device130 and a second, adjacent POS device 130, the resulting facial templatemay be associated with both the first merchant POS device 130 and thesecond merchant POS device 130.

If a facial template from the current customer log matches the generatedfacial template, the method 270 proceeds to block 1180. In anotherexample embodiment, if a facial template from the current customer logmatches a facial template from the log of facial templates of currentcustomers, the method 270 proceeds to block 1180

In block 1180, the merchant camera device 140 transmits the facialtemplate from the current customer log corresponding to the identifieduser to the payment processing system 160. For example, the transmittedfacial template comprises an annotation assigning the facial template toa particular one or two merchant POS devices 130.

From block 1180, the method 270 proceeds to block 295 in FIG. 2. In thisexample embodiment, the payment processing system 160 identifies theuser 101 requesting a transaction and a particular merchant POS device130 to which the user 101 is assigned and may proceed with processingthe transaction.

Returning to block 1170, if none of the facial templates from thecurrent customer log matches the generated facial template, the method270 proceeds to block 280 in FIG. 2. In another example embodiment, ifnone of the facial templates from the current customer log matches anyof the facial templates from the log of facial templates of currentcustomers, the method 270 proceeds to block 280 in FIG. 2.

Returning to block 280, in FIG. 2, the payment processing system 160identifies the user 101 via voice recognition. In another exampleembodiment, the payment processing system 160 does not identify the user101 via voice recognition and proceeds to identify the user 101 viachallenge and response. For example, the merchant camera device 140and/or payment processing system 160 is unable to identify the user viafacial recognition and/or if the merchant camera device 140 and/orpayment processing system 160 identify two or more users 101 via facialrecognition. The method for identifying, by a payment processing system160, a user 101 via voice recognition is described in more detailhereinafter with reference to the method 280 described in FIG. 12.

FIG. 12 is a block diagram depicting a method 280 for identifying, by apayment processing system 160, a user 101 via voice recognition, inaccordance with certain example embodiments. The method 280 is describedwith reference to the components illustrated in FIG. 1.

In certain example embodiments, it may be necessary to identify a user101 via voice recognition because the merchant camera device 140 and/orpayment processing system 160 is not able to identify the user 101 basedon facial recognition. For example, the merchant camera device 140 maybe unable to extract an adequate facial image of the user 101 from thevideo feed to generate a facial template. In another example, themerchant camera device 140 is unable to find a match for the facialtemplate of the user 101 in the current customer log. In yet anotherexample, the merchant camera device 140 identifies a matching user 101facial template in the current customer log, however, the identifieduser 101 facial template is assigned to two adjacent merchant POSdevices 130. In this example, the user 101 may have to identify himselfvia voice recognition at the POS device 130 at which the user 101initiates the transaction.

In block 1210, the merchant camera device 140 transmits a notice thatthe user 101 cannot be identified to the payment processing system 160.In another example embodiment, the merchant camera device 140 transmitsa notice that the user 101 has been identified, however, that themerchant camera device 140 is unable to determine which of two adjacentmerchant POS devices 130 the user 101 using in the transaction.

In block 1220, the payment processing system 160 retrieves audiotemplates corresponding to users 101 from the current customer log. Aspreviously discussed, when a user 101 establishes an account with thepayment processing system 160, the user 101 may submit an audiorecording of the user's 101 voice to the payment processing system 160.In this example the payment processing system 160 establishes an audiotemplate corresponding to the user 101 based on the received audiorecording. In another example embodiments, the user 101 does not submitan audio recording to the payment processing system 160 at the time theuser 101 establishes the user 101 account. In this other exampleembodiment, the payment processing system 160 does not have an audiotemplate associated with the user 101 account and cannot verify the user101 via voice recognition.

In block 1230, the payment processing system 160 transmits a request tothe merchant system point of sale device 130 to record audio of the user101. For example, the payment processing system 160 transmits therequest over the network 120.

In block 1240, the merchant system point of sale device 130 displays therequest for the user 101 to record audio. For example, the merchantsystem POS device 130 may display directions to the user 101 to recordan audio recording. For example, the user 101 may be directed to speakthe same words in the same intonation as the user 101 did whenestablishing the audio template with the payment processing system 160at the time of setting up the user 101 account.

In block 1250, the merchant system point of sale device 130 records avoice input of the user 101 and transmits the voice input to the paymentprocessing system 160. For example, the merchant POS device operator 102may actuate an object on the user interface 135 to activate an audiomodule 131 to receive an audio input of the user 101, which the merchantsystem POS device 130 retransmits to the payment processing system 160over the network 120. In an example embodiment, the merchant POS device130 transmits a merchant POS device 130 identifier along with theretransmitted user 101 audio input.

In block 1260, the payment processing system 160 compares the receivedvoice input against the retrieved audio templates corresponding to users101 from the current customer log. For example, the payment processingsystem 160 receives the audio input of the user 101 from the merchantPOS device 130 over the network 120.

In block 1270, the payment processing system 160 determines whetherthere is a match between the received voice input and one of theretrieved audio templates from the current customer log. In an exampleembodiment, the payment processing system 160 compares one or morefeatures between the received voice input and each of the retrievedaudio templates from the current customer log. In an example embodiment,if the similarity between the received voice input and a particularaudio template exceeds a predefined threshold, the payment processingsystem 160 determines that the received voice input matches theparticular audio template. In another example embodiment, if thesimilarity between the received voice input and the particular audiotemplate is less than the predefined threshold, the payment processingsystem 160 determines that the received voice input does not match theparticular audio template.

If an audio template from the current customer log matches the receivedvoice input, the method 270 proceeds to block 1280. For example, thesimilarity between a particular audio template from the current customerlog and the received audio recording of the user 101 exceeds apredefined threshold and the payment processing system 160 determinesthat there is a match.

In block 1280, the payment processing system 160 assigns the user 101 tothe merchant point of sale device 130 in the current customer log. Forexample, the payment processing system 160 identifies the user 101associated with the audio template that matches the received user 101audio recording. Additionally, the payment processing system 160identifies the merchant POS device 130 to which the user 101 should beassigned based on the merchant POS device 130 identifier received fromthe merchant POS device 130 with the user 101 audio recording.

From block 1290, the method 280 proceeds to block 295 in FIG. 2. Forexample, the payment processing system 160 successfully identifies theuser 101 based on comparing the received user 101 recording againstaudio templates from the current customer log. In this example, thepayment processing system 160 may continue to process the transactioninvolving the identified user 101.

Returning to block 1270, if none of the audio templates from the currentcustomer log matches the received voice input, the method 280 proceedsto block 290 in FIG. 2. In another example embodiment, the merchant POSdevice 130 is unable to receive a audio recording of the user 101 or thereceived audio recording of the user 101 is inadequate and cannot beused to identify the user 101.

Returning to FIG. 2, in block 290, the user 101 is identified by themerchant system POS operator 102 via a challenge and a response. Themethod for identifying, by a merchant point of sale device operator 102,a user 101 via a challenge and a response is described in more detailhereinafter with reference to the method 290 described in FIG. 13.

FIG. 13 is a block diagram depicting a method 290 for identifying, by amerchant point of sale device operator 102, a user 101 via a challengeand a response, in accordance with certain example embodiments. Themethod 290 is described with reference to the components illustrated inFIG. 1.

In block 1310, the point of sale device operator 102 issues a challengeto the user 101. In an example, the merchant POS device operator 102asks the user 101 for the initials of the user 101. In another example,the merchant POS device operator 102 asks the user 101 for the last fourdigits of the phone number of the user 101. In another example, themerchant POS device operator 102 asks the user 101 for a configuredpassword. Any suitable challenge may be issued by the merchant POSdevice operator 102. In an example embodiment, the response to thechallenge does not provide any secure or private information.

In block 1320, the user 101 provides a challenge response. As describedin the example challenges, the responses may be the initials of the user101, the last four digits of the phone number of the user 101, or aconfigured password. Any configured challenge response may be utilized.In certain embodiments, the response may be a spoken response, a handgesture, a keypad entry, a display of an identification card, or anysuitable response.

In block 1330, the point of sale device operator 102 inputs the responseinto the merchant point of sale device 130. The merchant POS deviceoperator 102 inputs the challenge response of the user 101. In anexample, if the user 101 indicates that the initials of the user 101 are“AC,” then the merchant POS device operator 102 inputs “AC” into thepayment application 133 of the merchant POS device 130. In an example,the user interface 135 of the merchant POS device 130 displays a requestfor an entry of the response of the user 101. The merchant POS deviceoperator 102 enters the response via a virtual or physical keyboard,voice dictation, or in any suitable manner. In an alternate example, theuser 101 enters the response into the user interface 135 of the merchantPOS device 130.

In block 1340, the merchant point of sale device 130 displays potentialusers 101 based on the challenge response. The merchant POS device 130displays potential users 101 based on the challenge response. A list ofusers 101 that are associated with the challenge response are displayedon the merchant POS device 130 to the merchant POS device operator 102.For example, if ten customers are in the vicinity of the merchant beacondevice 120, then the merchant POS device 130 may have received from thepayment processing system 160 a challenge response associated with eachof the respective ten customers. When the merchant POS device 130receives the challenge response input, only the potential users 101 thatare associated with the challenge response are displayed to the merchantPOS device operator 102.

In another embodiment, the merchant POS device 130 or the paymentprocessing system 160 which processes the challenge, presents additionalchallenges until there is a single matching user 101 remaining.

In the example, if the merchant POS device operator 102 inputs “AC” asthe initials of the user 101 associated with the transaction, then onlythe potential users 101 with those initials will be displayed to themerchant POS device operator 102 by the payment application 133. Thepayment application 133 accesses a database on the payment processingsystem 160 or another computing device and identifies the initials ofthe potential users 101 that have provided tokens. The paymentapplication 133 identifies the one or more potential users 101 that havethe initials “AC” and displays the identified user 101 accounts to themerchant POS device operator 102. In the example, two of the tencustomers that are in the vicinity of the merchant beacon device 120have the initials “AC.” The user 101 accounts of the two customers aredisplayed to the merchant POS device operator 102.

In certain example embodiments, all of the nearby customers who have hadtokens transmitted to the merchant POS device 130 are presented to themerchant POS device operator 102 and the merchant POS device operator102 selects the appropriate user 101 account.

The payment application 133 may display a picture of the potential user101 accounts that are presented to the merchant POS device operator 102.For example, each user 101 may associate a picture with a user 101account. When the merchant POS device 130 presents the one or morepotential user 101 accounts to the merchant POS device operator 102, themerchant POS device operator 102 may select the appropriate user 101account based on the picture matching the user 101 conducting thetransaction. Other identifying information may be presented instead of,or in addition to, a picture. For example, the name of the user 101 maybe displayed and the merchant POS device operator 102 may identify thepotential user 101 with that name. Any other suitable identifyinginformation may be presented.

In block 1350, the merchant point of sale device operator 102 selectsthe user 101 account for use in a transaction. After identifying thedisplayed picture of the user 101, the merchant POS device operator 102may input a selection of the user 101 by actuating a user interface 135control associated with the picture, or by inputting the selection inany suitable manner. If the picture doesn't match any of the potentialusers, then the merchant POS device operator 102 may cancel thetransaction, notify the user 101 of the discrepancy, or perform anyother suitable action.

In an example, only a single user 101 account is presented in the listof potential users 101. If only a single user 101 account is identified,then the method may proceed after the merchant POS device operator 102verifies that the displayed picture matches the user 101. If the picturedoesn't match, then the merchant POS device operator 102 may cancel thetransaction, notify the user 101 of the discrepancy, or perform anyother suitable action.

From block 1350, the method 290 proceeds to block 295 in FIG. 2.

Returning to FIG. 2, in block 295, a transaction is processed. Themethod for processing a transaction is described in more detailhereinafter with reference to the method 290 described in FIG. 14.

FIG. 14 is a block diagram depicting a method 295 for processing atransaction, in accordance with certain example embodiments. The method295 is described with reference to the components illustrated in FIG. 1.

In block 1410, the payment processing system 160 transmits accountinformation of the identified user 101 to the merchant point of saledevice 130. For example, the user 101 account with the paymentprocessing system 160 comprises payment account information associatedwith one or more payment accounts of the user 101. An example paymentaccount of the user 101 comprises a credit account, bank account, storedvalue account, gift card, merchant system charge account, or otherrelevant user 101 account. In an example embodiment, the merchant POSdevice 130 does not receive complete payment information for one or moreof the user 101 payment accounts. For example, the payment informationmay be obfuscated or truncated. For example, the merchant POS device 130may only receive abbreviated payment information sufficient for a user101 to identify a payment account via the user interface 135.

In block 1420, the merchant point of sale device 130 displays accountsof the identified user 101. In an example embodiment, the merchant POSdevice 130 does not display complete account information of the user 101on the user interface 135. For example, for a credit card account, themerchant POS device user interface 135 may display only the nameassociated with the credit card, last four digits of the credit cardnumber, and the expiration date.

In block 1430, the merchant point of sale device operator 102 selects auser 101 account for transaction and confirms the transaction withpermission of the user 101. For example, the user 101 views the accountoptions directly displayed on the user interface 135 or listens to themerchant POS device operator 102 read the user 101 payment accountoptions to the user 101. In an example embodiment, the user 101 selectsa payment account option to use in the transaction. For example, theuser 101 decides to use a credit card account. In an example embodiment,the user 101 communicates a selection of a payment account for use inthe transaction to the merchant POS device operator 102. In an exampleembodiment, in response to receiving the user's 101 indication of thepayment account, the merchant POS device operator 102 selects thepayment account indicated by the user 101 on the merchant POS deviceuser interface 135.

In block 1440, the merchant point of sale device 130 transmitstransaction details to the payment processing system 160. For example,the merchant POS device 130 transmits transaction details comprising atransaction total, an account identifier of the merchant system, and theselection of the user 101 account payment option. In an exampleembodiment, the payment processing system 160 receives the transactiondetails over the network 120. In an example embodiment, the paymentprocessing system 160 extracts payment account information from the user101 account corresponding to the received selection of the user 101payment account.

In block 1450, the payment processing system 160 transmits a transactionauthorization request to an issuer system 150. In an example embodiment,the payment processing system 160 generates a transaction authorizationrequest comprising user 101 payment account information, merchant systempayment account information, and a total amount of the transaction. Inan example embodiment, the payment processing system 160 receives thetransaction authorization request from the payment processing system 160via the network 120.

In block 1460, the issuer system 150 approves the transactionauthorization request and transmits a transaction authorization approvalto the payment processing system 160. In an example embodiment, theissuer system 150 subtracts the user's 101 current account balance fromthe user's 101 credit limit associated with the user's 101 creditaccount. In an example embodiment, if the different between the creditlimit and the credit balance is a positive number greater than the totalamount of the transaction, the issuer system 150 approves thetransaction authorization. For example, the user 101 will not exceed hiscredit limit if the transaction were to be approved, therefore theissuer system 150 approves the transaction authorization request. In anexample embodiment, the payment processing system 160 receives thetransaction authorization approval from the issuer system 150. Inanother example embodiment, the payment processing system 160 receives adenial of the transaction authorization request in response to theissuer system 150 declining the transaction authorization request. In anexample embodiment, if the issuer system 150 approves the transactionauthorization request, the issuer system 150 credits the merchant systemaccount and adds a charge to the user 101 credit account statement, bankaccount statement, or other appropriate statement.

In block 1470, the payment processing system 160 transmits a transactionreceipt to the merchant point of sale device 130. For example, thepayment processing system 160 transmits a confirmation of the approvalof the transaction or a notice of a declined transaction to the merchantPOS device 130 over the network 120. In an example embodiment, themerchant POS device 130 prints a receipt comprising a summary of asuccessful or unsuccessful transaction via a printer device that is acomponent of or otherwise communicatively coupled to the merchant POSdevice 130. In another example embodiment, the payment processing system160 may transmit a transaction receipt to the user computing device 110to be displayed on the user computing device. In yet another exampleembodiment, the merchant POS device 130 transmits a transaction receiptto the user computing device 110 to be displayed on the user computingdevice.

Other Example Embodiments

FIG. 15 depicts a computing machine 2000 and a module 2050 in accordancewith certain example embodiments. The computing machine 2000 maycorrespond to any of the various computers, servers, mobile devices,embedded systems, or computing systems presented herein. The module 2050may comprise one or more hardware or software elements configured tofacilitate the computing machine 2000 in performing the various methodsand processing functions presented herein. The computing machine 2000may include various internal or attached components such as a processor2010, system bus 2020, system memory 2030, storage media 2040,input/output interface 2060, and a network interface 2070 forcommunicating with a network 2080.

The computing machine 2000 may be implemented as a conventional computersystem, an embedded controller, a laptop, a server, a mobile device, asmartphone, a set-top box, a kiosk, a vehicular information system, onemore processors associated with a television, a customized machine, anyother hardware platform, or any combination or multiplicity thereof. Thecomputing machine 2000 may be a distributed system configured tofunction using multiple computing machines interconnected via a datanetwork or bus system.

The processor 2010 may be configured to execute code or instructions toperform the operations and functionality described herein, managerequest flow and address mappings, and to perform calculations andgenerate commands. The processor 2010 may be configured to monitor andcontrol the operation of the components in the computing machine 2000.The processor 2010 may be a general purpose processor, a processor core,a multiprocessor, a reconfigurable processor, a microcontroller, adigital signal processor (“DSP”), an application specific integratedcircuit (“ASIC”), a graphics processing unit (“GPU”), a fieldprogrammable gate array (“FPGA”), a programmable logic device (“PLD”), acontroller, a state machine, gated logic, discrete hardware components,any other processing unit, or any combination or multiplicity thereof.The processor 2010 may be a single processing unit, multiple processingunits, a single processing core, multiple processing cores, specialpurpose processing cores, co-processors, or any combination thereof.According to certain embodiments, the processor 2010 along with othercomponents of the computing machine 2000 may be a virtualized computingmachine executing within one or more other computing machines.

The system memory 2030 may include non-volatile memories such asread-only memory (“ROM”), programmable read-only memory (“PROM”),erasable programmable read-only memory (“EPROM”), flash memory, or anyother device capable of storing program instructions or data with orwithout applied power. The system memory 2030 may also include volatilememories such as random access memory (“RAM”), static random accessmemory (“SRAM”), dynamic random access memory (“DRAM”), and synchronousdynamic random access memory (“SDRAM”). Other types of RAM also may beused to implement the system memory 2030. The system memory 2030 may beimplemented using a single memory module or multiple memory modules.While the system memory 2030 is depicted as being part of the computingmachine 2000, one skilled in the art will recognize that the systemmemory 2030 may be separate from the computing machine 2000 withoutdeparting from the scope of the subject technology. It should also beappreciated that the system memory 2030 may include, or operate inconjunction with, a non-volatile storage device such as the storagemedia 2040.

The storage media 2040 may include a hard disk, a floppy disk, a compactdisc read only memory (“CD-ROM”), a digital versatile disc (“DVD”), aBlu-ray disc, a magnetic tape, a flash memory, other non-volatile memorydevice, a solid state drive (“SSD”), any magnetic storage device, anyoptical storage device, any electrical storage device, any semiconductorstorage device, any physical-based storage device, any other datastorage device, or any combination or multiplicity thereof. The storagemedia 2040 may store one or more operating systems, application programsand program modules such as module 2050, data, or any other information.The storage media 2040 may be part of, or connected to, the computingmachine 2000. The storage media 2040 may also be part of one or moreother computing machines that are in communication with the computingmachine 2000 such as servers, database servers, cloud storage, networkattached storage, and so forth.

The module 2050 may comprise one or more hardware or software elementsconfigured to facilitate the computing machine 2000 with performing thevarious methods and processing functions presented herein. The module2050 may include one or more sequences of instructions stored assoftware or firmware in association with the system memory 2030, thestorage media 2040, or both. The storage media 2040 may thereforerepresent examples of machine or computer readable media on whichinstructions or code may be stored for execution by the processor 2010.Machine or computer readable media may generally refer to any medium ormedia used to provide instructions to the processor 2010. Such machineor computer readable media associated with the module 2050 may comprisea computer software product. It should be appreciated that a computersoftware product comprising the module 2050 may also be associated withone or more processes or methods for delivering the module 2050 to thecomputing machine 2000 via the network 2080, any signal-bearing medium,or any other communication or delivery technology. The module 2050 mayalso comprise hardware circuits or information for configuring hardwarecircuits such as microcode or configuration information for an FPGA orother PLD.

The input/output (“I/O”) interface 2060 may be configured to couple toone or more external devices, to receive data from the one or moreexternal devices, and to send data to the one or more external devices.Such external devices along with the various internal devices may alsobe known as peripheral devices. The I/O interface 2060 may include bothelectrical and physical connections for operably coupling the variousperipheral devices to the computing machine 2000 or the processor 2010.The I/O interface 2060 may be configured to communicate data, addresses,and control signals between the peripheral devices, the computingmachine 2000, or the processor 2010. The I/O interface 2060 may beconfigured to implement any standard interface, such as small computersystem interface (“SCSI”), serial-attached SCSI (“SAS”), fiber channel,peripheral component interconnect (“PCI”), PCI express (PCIe), serialbus, parallel bus, advanced technology attached (“ATA”), serial ATA(“SATA”), universal serial bus (“USB”), Thunderbolt, FireWire, variousvideo buses, and the like. The I/O interface 2060 may be configured toimplement only one interface or bus technology. Alternatively, the I/Ointerface 2060 may be configured to implement multiple interfaces or bustechnologies. The I/O interface 2060 may be configured as part of, allof, or to operate in conjunction with, the system bus 2020. The I/Ointerface 2060 may include one or more buffers for bufferingtransmissions between one or more external devices, internal devices,the computing machine 2000, or the processor 2010.

The I/O interface 2060 may couple the computing machine 2000 to variousinput devices including mice, touch-screens, scanners, electronicdigitizers, sensors, receivers, touchpads, trackballs, cameras,microphones, keyboards, any other pointing devices, or any combinationsthereof. The I/O interface 2060 may couple the computing machine 2000 tovarious output devices including video displays, speakers, printers,projectors, tactile feedback devices, automation control, roboticcomponents, actuators, motors, fans, solenoids, valves, pumps,transmitters, signal emitters, lights, and so forth.

The computing machine 2000 may operate in a networked environment usinglogical connections through the network interface 2070 to one or moreother systems or computing machines across the network 2080. The network2080 may include wide area networks (WAN), local area networks (LAN),intranets, the Internet, wireless access networks, wired networks,mobile networks, telephone networks, optical networks, or combinationsthereof. The network 2080 may be packet switched, circuit switched, ofany topology, and may use any communication protocol. Communicationlinks within the network 2080 may involve various digital or an analogcommunication media such as fiber optic cables, free-space optics,waveguides, electrical conductors, wireless links, antennas,radio-frequency communications, and so forth.

The processor 2010 may be connected to the other elements of thecomputing machine 2000 or the various peripherals discussed hereinthrough the system bus 2020. It should be appreciated that the systembus 2020 may be within the processor 2010, outside the processor 2010,or both. According to some embodiments, any of the processor 2010, theother elements of the computing machine 2000, or the various peripheralsdiscussed herein may be integrated into a single device such as a systemon chip (“SOC”), system on package (“SOP”), or ASIC device.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity or option to control whetherprograms or features collect user information (e.g., information about auser's social network, social actions or activities, profession, auser's preferences, or a user's current location), or to control whetherand/or how to receive content from the content server that may be morerelevant to the user. In addition, certain data may be treated in one ormore ways before it is stored or used, so that personally identifiableinformation is removed. For example, a user's identity may be treated sothat no personally identifiable information can be determined for theuser, or a user's geographic location may be generalized where locationinformation is obtained (such as to a city, ZIP code, or state level),so that a particular location of a user cannot be determined. Thus, theuser may have control over how information is collected about the userand used by a content server.

Embodiments may comprise a computer program that embodies the functionsdescribed and illustrated herein, wherein the computer program isimplemented in a computer system that comprises instructions stored in amachine-readable medium and a processor that executes the instructions.However, it should be apparent that there could be many different waysof implementing embodiments in computer programming, and the embodimentsshould not be construed as limited to any one set of computer programinstructions. Further, a skilled programmer would be able to write sucha computer program to implement an embodiment of the disclosedembodiments based on the appended flow charts and associated descriptionin the application text. Therefore, disclosure of a particular set ofprogram code instructions is not considered necessary for an adequateunderstanding of how to make and use embodiments. Further, those skilledin the art will appreciate that one or more aspects of embodimentsdescribed herein may be performed by hardware, software, or acombination thereof, as may be embodied in one or more computingsystems. Moreover, any reference to an act being performed by a computershould not be construed as being performed by a single computer as morethan one computer may perform the act.

The example embodiments described herein can be used with computerhardware and software that perform the methods and processing functionsdescribed herein. The systems, methods, and procedures described hereincan be embodied in a programmable computer, computer-executablesoftware, or digital circuitry. The software can be stored oncomputer-readable media. For example, computer-readable media caninclude a floppy disk, RAM, ROM, hard disk, removable media, flashmemory, memory stick, optical media, magneto-optical media, CD-ROM, etc.Digital circuitry can include integrated circuits, gate arrays, buildingblock logic, field programmable gate arrays (FPGA), etc.

The example systems, methods, and acts described in the embodimentspresented previously are illustrative, and, in alternative embodiments,certain acts can be performed in a different order, in parallel with oneanother, omitted entirely, and/or combined between different exampleembodiments, and/or certain additional acts can be performed, withoutdeparting from the scope and spirit of various embodiments. Accordingly,such alternative embodiments are included in the scope of the followingclaims, which are to be accorded the broadest interpretation so as toencompass such alternate embodiments.

Although specific embodiments have been described above in detail, thedescription is merely for purposes of illustration. It should beappreciated, therefore, that many aspects described above are notintended as required or essential elements unless explicitly statedotherwise. Modifications of, and equivalent components or actscorresponding to, the disclosed aspects of the example embodiments, inaddition to those described above, can be made by a person of ordinaryskill in the art, having the benefit of the present disclosure, withoutdeparting from the spirit and scope of embodiments defined in thefollowing claims, the scope of which is to be accorded the broadestinterpretation so as to encompass such modifications and equivalentstructures.

What is claimed is:
 1. A computer-implemented method to identify usersat locations by comparing facial imaging of users against facialtemplates of users known to be at particular locations, comprising: by acomputing device: receiving, from one or more computing devices, one ormore facial templates, each facial template associated with acorresponding user associated with a corresponding user computing devicethat received a broadcasted identifier at a location and retransmittedthe identifier to one or more other computing devices, each facialtemplate comprising a representation of a respective facial image of therespective user; capturing a facial image of the particular user;generating a facial template of the particular user based on the facialimage of the particular user, wherein the facial template of theparticular user is a representation of the facial image of theparticular user; determining a similarity between the generated facialtemplate and each of the one or more stored facial templates;identifying a particular received facial template as corresponding tothe generated facial template of the particular user based on thedetermined similarity between the generated facial template and theparticular received facial template exceeding a threshold valueindicating that the received facial template corresponds to thegenerated facial template of the particular user; and transmitting aresponse to a request to identify a user comprising an indication of theidentified particular user.
 2. The method of claim 1, further comprisingreceiving, by the computing device, a request to identify the user. 3.The method of claim 1, wherein capturing the facial image of theparticular user comprises: capturing, by a camera module of thecomputing device, a video feed of an environment external to thecomputing device, wherein the environment external to the computingdevice comprises a user located in proximity to one or more merchantsystem point of sale devices; and extracting, by the computing device,the facial image of the particular user from the video feed.
 4. Themethod of claim 2, wherein the request to identify the user is receivedfrom a merchant system point of sale device, and wherein the response tothe request to identify the user is transmitted to the merchant point ofsale device.
 5. The method of claim 4, further comprising, processing,by the merchant point of sale device, a transaction using paymentaccount information associated with the identified user.
 6. The methodof claim 3, further comprising, by the computing device: determining,based on a frame of the video feed, a position of a face of the user andpositions of each of the one or more point of sale devices at themerchant location; and assigning, based on distances between the face ofthe user and each of the one or more point of sale devices in the frameof the video feed, the user to a particular point of sale device of theone or more point of sale devices, wherein the indication of theidentity of the user further comprises an identifier of the particularpoint of sale device and an indication of the assignment of the user tothe particular point of sale device.
 7. A computer-implemented method toidentify users at locations by comparing facial imaging of users againstfacial templates of users known to be at particular locations,comprising: by one or more computing devices: receiving, from a usercomputing device, an account identifier of a user and a merchant beaconidentifier, wherein the user computing device receives the merchantbeacon identifier from a merchant beacon device located at a location ofa merchant system; identifying a merchant system location associatedwith the merchant beacon identifier; adding a facial templatecorresponding to the account identifier of the user and the merchantsystem location to a current customer log comprising facial templates ofusers currently at the location of the merchant system; andtransmitting, to a merchant computing device at the location of themerchant system, the current customer log, wherein the merchantcomputing device identifies the user based on identifying a facialtemplate from the current customer log that is similar to a facialtemplate generated from a facial image of the user received by themerchant computing device.
 8. The method of claim 7, further comprising,by the one or more computing devices: determining that the usercomputing device is no longer in proximity of the merchant beacondevice; and removing the facial template of the user and the merchantsystem location from the current customer log.
 9. The method of claim 8,wherein the one or more computing devices determines that the usercomputing device is no longer in proximity of the merchant beacon deviceif a subsequent transmission of the user account identifier and themerchant beacon identifier is not received within a threshold amount oftime from the user computing device.
 10. The method of claim 7, furthercomprising, by the one or more computing devices: receiving, from thecomputing device, a facial template associated with an identified user,a transaction total, and a merchant account identifier from the merchantcomputing device at the location of the merchant system; determining,based on the received facial template, an account identifier associatedwith the user; transmitting, to a point of sale device, one or morepayment account information items associated with the user account;receiving, from the point of sale device, an indication of a selectionof a particular payment account information item for use in atransaction; generating a transaction authorization request comprisingthe transaction total, the merchant account identifier, and theparticular payment account information item; transmitting thetransaction authorization request to an issuer system associated withthe particular payment account information item; receiving, from theissuer system, data comprising an authorization of the transactionauthorization request; and transmitting, to the point of sale device fordisplay on the point of sale device, a message regarding authorizationof the transaction.
 11. The method of claim 10, further comprising,receiving, from the computing device, an assignment of the identifieduser to a particular point of sale device at the location of themerchant system, wherein the one or more payment account information aretransmitted to the particular point of sale device based on theassignment.
 12. A computer program product to identify users atlocations by comparing facial imaging of users against facial templatesof users known to be at particular locations, comprising: anon-transitory computer-readable medium having computer-readable programinstructions embodied thereon that when executed by a computer cause thecomputer to: receive, from one or more computing devices, one or morefacial templates, each facial template associated with a correspondinguser associated with a corresponding user computing device that receiveda broadcasted identifier at a location and retransmitted the identifierto one or more other computing devices, each facial template comprisinga representation of a respective facial image of the respective user;capture a facial image of the particular user; generate a facialtemplate based on the facial image of the particular user, wherein thefacial template of the particular user is a representation of the facialimage of the particular user; determine a similarity between thegenerated facial template and each of the one or more stored facialtemplates; identify a particular received facial template ascorresponding to the generated facial template of the particular userbased on the determined similarity between the generated facial templateand the particular received facial template exceeding a threshold valueindicating that the received facial template corresponds to thegenerated facial template of the particular user; and transmit aresponse to a request to identify a user comprising an indication of theidentified particular user.
 13. The computer program product of claim12, wherein capturing the facial image of the particular user comprises:capturing, by a camera module of the computing device, a video feed ofan environment external to the computing device, wherein the environmentexternal to the computer comprises a user located in proximity to one ormore merchant system point of sale devices; and extracting, by thecomputing device, the facial image of the particular user from the videofeed.
 14. The computer program product of claim 12, wherein the requestto identify the user is received from a merchant system point of saledevice and wherein the response to the request to identify the user istransmitted to the merchant point of sale device.
 15. The computerprogram product of claim 13, wherein the merchant point of sale deviceprocesses a transaction using payment account information associatedwith the identified user at a time after the request to identify theuser is transmitted to the merchant point of sale device.
 16. Thecomputer program product of claim 13, wherein the non-transitorycomputer-readable medium further comprises computer-readable programinstructions embodied thereon that when executed by the computer causethe computer to: determine, based on a frame of the video feed, aposition of a face of the user and positions of each of one or morepoint of sale devices at the merchant location; and assign, based ondistances between the face of the user and each of the one or more pointof sale devices in the frame of the video feed, the user to a particularpoint of sale device of the one or more point of sale devices, whereinthe indication of the identity of the user further comprises anidentifier of the particular point of sale device and an indication ofthe assignment of the user to the particular point of sale device.
 17. Asystem to identify users at locations by comparing facial imaging ofusers against facial templates of users known to be at particularlocations, comprising: a storage device; and a processor communicativelycoupled to the storage device, wherein the processor executesapplication code instructions that are stored in the storage device tocause the system to: receive, from a user computing device, an accountidentifier of a user and a merchant beacon identifier, wherein the usercomputing device receives the merchant beacon identifier via a networkconnection with a merchant beacon device located at a location of amerchant system; identify a merchant system location associated with themerchant beacon identifier; add a facial template corresponding to theaccount identifier of the user and the merchant system location to acurrent customer log comprising facial templates of users currently atthe location of the merchant system; and transmit, to a computing deviceat the location of the merchant system, the current customer log,wherein the computing device identifies the user based on identifying afacial template from the current customer log that is similar to afacial template generated from a received facial image of the user. 18.The system of claim 17, wherein the processor is further configured toexecute computer-readable program instructions stored on the storagedevice to cause the system to: determine that the user computing deviceno longer maintains a network connection with the merchant beacondevice; and remove the facial template of the user and the merchantsystem location from the current customer log.
 19. The system of claim17, wherein determining that the user computing device no longermaintains the network connection with the merchant beacon devicecomprises determining that a subsequent transmission of the user accountidentifier and the merchant beacon identifier is not received within athreshold amount of time from the user computing device.
 20. The systemof claim 17, wherein the processor is further configured to executecomputer-readable program instructions stored on the storage device tocause the system to: receive, from the computing device, a facialtemplate associated with an identified user, a transaction total, and amerchant account identifier from the computing device at the location ofthe merchant system; determine, based on the received facial template,an account identifier associated with the user; transmit, to a point ofsale device, one or more payment account information associated with theuser account; receive, from the point of sale device, an indication of aselection of a particular payment account information for use in atransaction; generate a transaction authorization request comprising thetransaction total, the merchant account identifier, and the particularpayment account information; transmit the transaction authorizationrequest to an issuer system associated with the particular paymentaccount; receive, from the issuer system, data comprising an approval ofthe transaction authorization request; and transmit, to the point ofsale device for display on the point of sale device, a receiptcomprising a summary of the transaction.